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ABSTRACT 



The Austin Collaborative for Mathematics Education (ACME) is 



a systemwide initiative to improve mathematics education in all elementary 
and middle school classrooms in the Austin Independent School District 
(AISD) , Texas. The initiative, funded by the National Science Foundation, 
provides long-term, high quality professional development to build the 
instructional capacity of more than 2,000 AISD mathematics teachers. This 
evaluation describes ACME professional development in its second year of 
implementation. It focuses on the roles and teamwork of ACME professional 
development facilitators, teachers' experiences in ACME professional 
development, effects on teachers' knowledge and skills, and implementation of 
the ACME. Data were gathered through: (1) questionnaires completed by 237 

teachers and 83 principals ; (2 ) interviews with 10 teachers , as well as 

administrators and ACME staff; (3) observations of professional development 
and 50 classroom mathematics lessons; and (4) examination of district 
documents . A key ACME strength was the way ACME core members work together 
and their vision of high quality standards-based professional development. 
ACME professional development provides many opportunities for teachers who 
engage in its activities actively. Some challenges were identified, mainly in 
finding ways to improve teachers' participation within the professional 
development sessions. Appendixes contain the teacher questionnaire, the 
teacher interview form, the principal questionnaire, the classroom 
observation guide and protocol, the post-observation interview guide, and 
other surveys and interview guides. (Contains 12 references.) (SLD) 
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Austin Independent School District 



Executive Summa^ 



T^he Austin Collaborative for Mathematics Education (ACME) is a systemwide initiative to improve 
^ mathematics education in all elementary and middle school classrooms in the Austin Independent School 
District (AISD). This initiative, funded by the Natipnal Science Foundation (NSF) and the district, provides long- 
term, high quality professional development to build the instructional capacity of over 2000 AISD mathematics 
teachers. ACME professional development supports teachers as they implement the district’s curriculum resources 
of Investigations in Number, Data, and Space and Connected Mathematics (CMP), which are aligned with the 
state standards for mathematics education in the Texas Essential Knowledge and Skills (TEKS) and the national 
standards set by the National Council of Teachers of Mathematics (NCTM). These standards focus on broadening 
the topics taught at all grade levels, developing children’s mathematical thinking, and deepening children’s 
conceptual understanding through concrete experiences (Russell, 1998). The standards contrast with traditional 
mathematics education characterized by rote memorization and computation practice. 

District staff design ACME professional development to help teachers grow as a community of learners 
and to deepen their knowledge of mathematics content, pedagogy, and classroom management of standards-based 
mathematics instruction. AISD administrators expect every elementary and middle school mathematics teacher, 
including general education, special education, bilingual, and English as a Second Language (ESL) teachers, to 
participate in a coordinated series of ACME professional development activities. These activities include two 
years of summer institutes and follow-up days during the academic year. 

To promote districtwide change in mathematics education, the ACME project bolsters leadership and the 
development of school cultures in which communities continually improve mathematics teaching and learning. 
ACME staff provide institutes for campus administrators to build knowledge of standards-based mathematics 
curriculum resources and instruction and to help campus leaders develop strategies for supporting teachers in 
implementation. ACME staff also work with other organizational structures in AISD that promote teacher 
leadership (e.g., curriculum specialists) to support the continuous improvement of mathematics education on 
campuses. In addition, the ACME project has customized professional development for teacher leaders so that 
they may facilitate sessions and support their peers on the campus level in a variety of ways, including peer 
coaching, demonstration teaching, and information sharing. To gamer parent participation in the mathematics 
curriculum j the project staff provides schools with technical support (e.g., pamphlets and videos in English and 
Spanish) as well as assistance with organizing parent education and involvement (e.g., parent math nights). 
Additionally, the project staff enlists support from AISD’s administrative leaders. 

The purpose of this evaluation is to describe ACME professional development in its second year of 
implementation and to document changes since its inception. In particular, the evaluation focuses on (a) the roles 
and teamwork of ACME professional development facilitators, (b) teachers’ experiences in ACME professional 
development, (c) effects on teachers’ knowledge and skills, and (d) implementation of ACME. 

Data were gathered through questionnaires completed by teachers and principals; interviews with 
teachers, ACME staff, and district administrators; observations of ACME professional development and of 50 
mathematics lessons in AISD elementary and middle school classrooms; and examination of district documents. 

ACME Professional Development Facilitators 

Five categories of facilitators provided ACME professional development in the 1998-1999 school year 
and summer of 1999. The core ACME team provided the bulk of support to teachers and direction for the 
initiative. District mathematics staff supported the initiative by giving feedback to the core ACME team about the 
day to day realities of implementation and by facilitating ACME summer institutes. Facilitators from Michigan 
schools who piloted CMP in their classrooms provided summer institutes to middle school mathematics teachers. 
A consultant with Marilyn Bums Education Associates facilitated several sessions for a cadre of 30 elementary and 
middle school teachers. Elementary teachers who had participated in the cadre modeled lessons from 
Investigations for participants at ACME summer institutes. 

Responsibilities of ACME team 

The core ACME team provided districtwide leadership for the initiative. The main responsibilities of the 
ACME team were to design and implement ACME professional development, create resources for teachers such as 
assessment and planning tools, and organize materials for professional development. They developed collegial 
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relationships with AISD teachers and administrators who participated in ACME professional development to 
support implementation. In addition, ACME team members provided campus support to eight pilot elementary 
schools, which involved modeling lessons, meeting with grade level teams, co-teaching, and mentoring teachers. 
Although some of these interactions were brief encounters that contained moral and professional support for 
implementation, the ACME team developed strategies that combined observation and professional conversations 
about children’s thinking and pedagogy to help teachers improve their skills in standards-based instruction. 
Teamwork 

The ACME team addressed their own professional development as facilitators by attending conferences, 
sharing knowledge with team members through collaboration, and bridging on the expertise of others. Generally, 
team members improved their skills as facilitators. The ACME team reported that teamwork and collaboration 
were their primary source of support. The fuel for this teamwork was (a) a shared vision of reform in mathematics 
education through high quality professional development for teachers, meaningful mathematics for children, and 
the transformation of teaching culture into systemic, professional collaboration and (b) strong team leadership with 
a drive to constantly improve the quality of the work. The ACME team also received support from ample funding, 
a network of district specialists, and central office administrators, although support from campus administrators 
was mixed. ACME facilitators have become less defensive to the opposition to the initiative and more responsive 
to the needs of teachers who participate in professional development than before. 

I Recommendations for ACME professional development facilitators: 

I : Continue to develop team support through collaboration and the drive to improve the work of ACME. 

j i • Continue to enlist tiie support of campus admiiiistrators for standards-based mathematics education. 

• Continue to address teachers’ needs through responsiveness, but evaluate priorities. 

Teachers^ Experiences in ACME Professional Development 

Opportunities for Teachers 

A key characteristic of ACME professional development was that facilitators designed sessions to address 
teachers’ needs by providing sessions, for example, on how to plan lessons and model teaching investigations. 
This approach made teachers feel that facilitators improved follow-ups by listening to teachers. Teachers 
considered opportunities to share with their colleagues on campus and across the district to be an asset of ACME 
professional development. Some teachers had lively discussion about the implementation of standards-based 
curriculum resources, pedagogy, and mathematics content. Participating in ACME professional development also 
provided the opportunity for teachers to discuss standards-based pedagogy, content knowledge, and ways to foster 
children’s thinking with ACME facilitators. Teachers also had opportunities to discuss and voice their concerns 
about district policies and practices concerning mathematics education at ACME sessions. 

Teachers' Engagement in ACME Activities 

The engagement of teachers in ACME professional development activities was not 100%. In 
observations, one-fourth to one^ third of the tables had teachers who were not actively involved in the activities. 
Teachers who do not actively engage may be disgruntled or reluctant to participate in an initiative that they do not 
endorse. Some may be reticent in large groups, especially concerning topics new to them. Within ACME 
professional development sessions, facilitators may tackle low teacher engagement by communicating flexibility 
and expectations. Flexibility can create a context in which participants feel welcome to engage when they are 
ready and in whatever formats with which they feel comfortable. Facilitators’ communicating the expectation that 
teachers will engage in activities within the session by asking probing questions and redirecting colleagues is 
another strategy for increasing what teachers take from sessions. 

Other ACME Professional Development 

Two professional development formats, campus support and teacher cadre, were offered to a small 
proportion of teachers who highly valued the opportunities. Teachers appreciated the mentoring and opportunities 
to talk with experts in standards-based instruction that campus support warranted. Although these interactions may 
pull teachers out of the isolation of teaching, it is important to note that reflective practice is necessary to improve 
instruction optimally. At teacher cadre meetings, teachers skilled in standards-based instruction could form 
professional relationships and engage in reflective discussions with similarly skilled colleagues. One shortcoming 
of these two professional development formats was that they touched few teachers in the district. 
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Recommendations for teachers’ exoeriences in ACME professionai development; 

. - Continue to provide high quality professional development in which teachers have oppormnities to share • 
with colleagues and experts in standards-based mathematics education. 

. Address teachers’ inactivitj- in professional development by asking engaging questions and communicating . 
the expectation that teachers will pamcipate. « • u ^ 

• Continue to develop strategies of campus support that promote meaningful discussions and reflecnon about . 
standards-based pedagogy that will help teachers improv e instruction. 

Effects on Teachers^ Knowledge and Skills in Standards-Based Mathematics 

Classroom Observations of Mathematics Instruction 

Mathematics lessons were rated using the Classroom Observation Protocol (HRI, 1999b) on an 8-point 
scale ranging from ineffective to effective standards-based instruction (see definitions of ratings on p. 18 of report). 
Statistical analyses revealed less variability in the distribution of observation ratings and a trend of more effective 
instruction for trained teachers (i.e., 
teachers (i.e., less than of ACME 
professional development). 

However, number of professional 
development hours for teachers 
was not ’ correlated with the 
observation ratings, which may be 
due to the different skills in 
standards-based instructions 
teachers have before they begin 
ACME. Features distinguishing 
effective and ineffective 
instruction were mathematics 
content knowledge and classroom 
culture. 

Improving Teachers* Knowledge of Mathematics Content 

ACME facilitators helped teachers improve their mathematics content knowledge by designing engaging 
problems for adult learners to solve that push teachers’ understanding to higher levels. The ACME team also 
designed activities in which teachers develop their own computational strategies, analyze mathematics concepts in 
students’ work, and encounter various topics such as measurement, algebra, and geometry infused throughout 
professional development activities. Although the ACME team tailored professional development activities to 
improve content knowledge, teachers did not have words to describe the content that they gained, especially 
teachers in elementary schools. Moreover, fewer teachers reported improvements in their mathematics content 
knowledge than did improvements in pedagogy and in the use of instructional materials. Assessment of the 
mathematics content knowledge that teachers gain in ACME professional development is lacking. 

Improving Teachers* Knowledge of Pedagogy 

To help teachers improve their knowledge of standards-based pedagogy and understand children’s 
thinking, the ACME team launched various conversations in professional development. Facilitators modeled open- 
ended questioning strategies and asked teachers to talk about the approach, led book studies about pedagogy, 
discussed the instructional strategies and children’s thinking in videos of AISD teachers skilled in standards-based 
instruction, and welcomed educators to reflect on their own teaching. The team also designed performance 
assessment rubrics and helped teachers learn how to implement them. These activities may be impacting teachers’ 
knowledge of pedagogy. Teachers reported more standards-based instruction in their classrooms than in the 
previous academic year and articulately discussed the strategies. However, pedagogical knowledge does not 
necessarily transfer to pedagogical skills. Classroom observations of mathematics lessons taught by teachers who 
participated in ACME professional development reflected a range of skills in standards-based instruction. 

Teachers * Learning How to Use the Instructional Materials 

In the 1999-2000 academic year, all mathematics teachers will have their own standards-based curriculum 
resources, kit of manipulatives, and copies of student sheets because the AISD board of trustees agreed to fund 
these needs of teachers. ACME professional development activities that helped teachers learn how to use these 
materials included: (a) engaging in a scavenger hunt through the resources; (b) exploring games and the 
underlying mathematical concepts; (c) discussing investigations in follow-up sessions to teach in subsequent 
months; and (d) observing classroom teachers model teach lessons from the resources. What teachers valued about 
ACME professional development were the instructional materials and support for learning how to use them. Over 




half of the teachers surveyed reported that participation increased their ability to implement high quality 
instructional materials, although some teachers found the resources repetitive and not meaningfi^l. 



I Recommendations for improving teachers' knowledge and skills in siandards-based mathematics: 
j V • Focnis on helping teachers learn more :mathematics content knowledge; determine standards with which I 
teachers are uncomfortable, set goals for tackling those concepts in professional development, and assess : 
how well those goals were met 

• Continue to help teachers gain standards-based pedagogical knowledge: set up peer coaching networks ;bn i 
campuses to provide teachers observation and feedback on their instructional strategies, 
j • Continue to actively engage teachers with instructional materials and to have classroom teachers model ^ 
j . lessons. 



Implementation of ACME Professional Development 

The ACME professional development model is on course such that all teachers of elementary and middle 
school grade levels will have the opportunity to participate by the Spring of 2002 when the NSF grant terminates. 
Attendance at summer and follow-up institutes in 1999 was less than 100%, with 80% to 90% of the teachers 
expected to attend participating in the first week and 75% participating in the second week of the institutes. High 
turnover in the district (i.e., about 60 new elementary teachers and about 20 new middle school teachers each year) 
and teachers’ changing grade levels from year to year influenced the ACME staff to repeat institutes for all grade 
levels every summer. Because some new hires and other teachers missed summer institutes that they were targeted 
to attend, ACME staff held brief one-day overviews about standards-based mathematics in the first month of 
school. This professional development system appears to reach a majority of AISD mathematics teachers, not all. 

i Recommendations for sustaining professional development in standards-based mathematics: . : : j 

I ■ • Establish summer institutes, follow-up during the academic year, and overviews for new hires at all grade | 
! levels that will continue after the NSF grant ends. 
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Overview 

In August of 1997, the Austin Independent School District (AISD) launched the Austin 
Collaborative for Mathematics Education (ACME) initiative to improve mathematics education 
in all elementary and middle school classrooms with standards-based curriculum resources and 
instruction. The National Science Foundation (NSF) funded the initiative, which receives 
support from the Charles A. Dana Center and the University of Texas at Austin. In the 1998-99 
school year, the ACME project served over 2000 AISD educators who teach about 55,000 
students at 68 elementary and 15 middle schools in a district of approximately 77,000 students 
(43% Hispanic, 18% African American, 37% Anglo and other; 50% receive free or reduced 
lunch and 13% receive bilingual or ESL services; AISD Office of Student Services, Sept. 1999). 
The ACME project is unique because it serves every elementary and middle school mathematics 
teacher in a large urban district with long-term professional development. 

The ACME project builds the instructional capacity of all mathematics teachers by 
providing a minimum of 150 hours of professional development through summer institutes and 
follow-up sessions. Some teachers also participate in campus level support such as peer 
coaching, lesson modeling, and collaborative planning. The intent of ACME professional 
development is to build teachers’ capacity to deliver effective mathematics instruction to all 
students, to ensure consistent implementation of quality mathematics curriculum resources across 
the district, and to provide ongoing support for teachers and administrators as they implement 
standards-based curriculum and instruction. Specifically, district staff design ACME 
professional development to help teachers grow as a community of learners and to deepen their 
knowledge of mathematics content, pedagogy, and classroom management for inquiry-based 
mathematics instruction. 

AISD administrators expect every elementary and middle school mathematics teacher, 
including general education, special education, bilingual, and English as a Second Language 
(ESL) teachers, to participate in a coordinated series of ACME professional development 
activities. Participants begin their training with a summer institute lasting two weeks and 
continue with four to five follow-up days during the academic year. The second phase involves a 
three-day summer institute and three to four follow-up days. Teachers are paid a stipend to 
attend the summer institutes and follow-up sessions outside school hours, and substitutes are 
provided to release teachers during the academic year. To accommodate the needs of AISD 
teachers and administrators, ACME staff adjusted the original design of ACME by adding 
professional development sessions on Saturdays and evenings, designing sessions for special 
education teachers, and adding overviews for late hires. To address teacher turn-over (more than 
500 new hires yearly), ACME staff repeated summer institutes for each grade level. 

ACME professional development began working with teachers at the transition between 
elementary and middle school so that students would have consistent mathematics instruction 
from one year to the next. In the summer of 1997, fifth and sixth grade teachers began ACME 
professional development, followed by fourth and seventh grade teachers in the summer of 1998, 
second, third, and eighth grade teachers in the summer of 1999. Although kindergarten and first 
grade teachers will participate in summer institutes in the summer of 2000, ACME staff will 
introduce the standards-based curriculum resources to teachers of these grade levels in the fall of 
1999 (because the resources are now the district’s adopted texts). At most schools in the district, 
AISD is implementing ACME professional development by grade levels. Yet, at eight pilot 
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elementary schools, teachers of all grade levels participated in ACME professional development 
simultaneously. Three pilot middle schools participated in the NSF-funded State Systemic 
Initiative (SSI) beginning with sixth grade mathematics teachers in the summer of 1996. Pilot 
schools receive modified summer institutes: fewer days of summer institutes and follow-up 
sessions in exchange for on campus support such as modeling lessons and conversations about 
curriculum and instruction. 

To support the mathematics instructional capacity of teachers, the district is implementing 
rigorous curriculum resources as part of the ACME initiative. The resources are based on 
standards set by the National Council of Teachers of Mathematics (NCTM, 1989, 1991, 1995), 
by the state in the Texas Essential Knowledge and Skills (TEKS), and by AISD’s Mathematics 
Department in the local curriculum document. In the spring of 1999, the district adopted the 
curriculum resources of Investigations in Number, Data, and Space for elementary grades and 
Connected Mathematics (CMP) for middle grades, and purchased these materials to support 
teachers’ implementation of standards-based instruction. AISD also adopted the resources of 
Math in My World (English version)/ Mathematicas in Mi Mundo (Spanish version) for 
elementary grades and Mathematics: Applications and Connections, Courses 1-3 (English 
version)/ Mathematicas: Aplicaciones y Coneciones, Cursos 1-3 (Spanish version) to supplement 
TEKS areas not addressed in Investigations and CMP. This adoption ensures that all of AISD’s 
mathematics education resources and efforts are aligned with local, state, and national standards. 

These curriculum resources of Investigations and CMP are particularly well suited for 
AISD because they support the following standards-based teaching practices: 

• Promoting children’s mathematical thinking, reasoning, and problem-solving skills; 

• Developing children’s deep understanding of mathematical concepts through concrete 
experiences, real-world problems, and communication; and 

• Supporting a vertically and horizontally coordinated curriculum that addresses the 
needs of all students, including those who are special education, limited English 
proficient, bilingual, and gifted and talented (Russell, 1998). 

These practices emphasize children’s mathematical literacy by promoting the 
understanding of mathematics concepts and approach instruction through problem-solving and 
communication of ideas. These practices contrast with traditional practices that emphasize 
mathematical algorithms, rote memorization, and computation mastery (Cohen & Ball, 1990). 

To promote districtwide change in mathematics education, the ACME project bolsters 
leadership and the development of school cultures in which communities continually improve 
mathematics teaching and learning. ACME staff provide institutes for campus administrators to 
build knowledge of standards-based mathematics curriculum resources and instruction and to 
help campus leaders develop strategies for supporting teachers in implementation. ACME staff 
also work with other organizational structures in AISD that promote teacher leadership (e.g., 
curriculum specialists) to support the continuous improvement of mathematics education on 
campuses. In addition, the ACME project has customized professional development for teacher 
leaders so that they may facilitate sessions and support their peers on the campus level in a 
variety of ways, including peer coaching, demonstration teaching, and information sharing. To 
gamer parent participation in the mathematics curriculum, the project staff provides schools with 
technical support (e.g., pamphlets and videos in English and Spanish) as well as assistance with 
organizing parent education and involvement (e.g., parent math nights). Additionally, the project 
staff enlists support from AISD’s administrative leaders. 
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Evaluation Design 

The purpose of this evaluation is to describe ACME professional development in its 
second year of implementation and to document changes since its inception. The information 
was gathered according to the evaluation design of Horizon Research, Inc. (HRI) and 
supplemented with other structured and unstructured data gathered locally (Batchelder & 
Christian, 1999). The major sources of information included AISD mathematics teachers, 
principals, central office administrators, district mathematics staff, and documents as well as 
observation ratings. 

Teacher Questionnaires 

A random sample of 300 AISD elementary and middle school mathematics teachers were 
sent questionnaires. Twenty-four teachers were omitted from the sample because they were not 
currently teaching mathematics in the classroom (i.e., they taught language arts and social studies 
only, were on personal leave, or had resigned). Of the 276 eligible teachers, 237 returned valid 
questionnaires with a return rate of 86%. For these 237 teachers, most (88%) were female, more 
than half were White (59% White, 30% Hispanic, 7% African American, 1% Asian or Pacific 
Islander, and 4% other or unknown), most (95%) had completed 2 semesters or more of college 
mathematics but only some (23%) had completed 5 semesters or more, and about one-third 
(34%) had taught school for 5 years or less, one-third (29%) had taught for 6 to 15 years, and 
one-third (37%) had taught for 16 years or more. 

For the purposes of this evaluation, teachers were defined as trained when they had 
attended at least 20 hours of ACME professional development (HRI, 1999a). According to 
questionnaire responses, half of the teachers surveyed were trained and half were untrained. 

The HRI questionnaires surveyed teachers’ beliefs about mathematics instruction, 
preparation, classroom practice, mathematics content knowledge, perceptions of district support, 
and experiences in ACME professional development (see Appendix). The results of the 1997-98 
survey suggested that teachers did not distinguish ACME professional development from other 
professional development provided by the district (i.e., teachers reported more hours of 
professional development than ACME had offered at the time). Consequently, to improve the 
reliability of responses, each questionnaire was affixed with a statement specifying that items 
about professional development referred to “ACME training in Investigations and CMP." 

Teacher Interviews 

Additional information about the project was gathered through interviews with 10 
mathematics teachers randomly drawn from the previous sample of 300 teachers. (The 
evaluation design initiated these interviews in the project’s second year.) The 10 randomly 
selected teachers taught 2"^ through 5* grades, and half were 4* grade teachers. All 10 of these 
teachers were trained, and all but one had participated in 30 or more hours of ACME professional 
development. The lead evaluator conducted all interviews over the phone. The interviews, 
designed by HRI, included teachers’ thoughts and feelings about ACME professional 
development, the presence or absence of changes in their teaching practice, their needs to 
improve mathematics instruction, and school and district policies that facilitate or hinder reforms 
in mathematics education (see Appendix). 
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Principal Questionnaires 

In the spring of 1998, all AISD principals of the 15 middle schools and the 68 elementary 
schools completed questionnaires about standards-based mathematics and science curricula, 
ACME professional development, and school characteristics (see Appendix). Forty-two percent 
of the respondents had held the position of principal for 3 years or less and about two-thirds of 
the respondents had held the position of principal at the school or in AISD for 3 years or less. As 
with the teacher questionnaires, each principal questionnaire included a statement specifying that 
items about professional development referred to “ACME training in Investigations and CMP." 

Classroom Observations 

From the sample of 300 mathematics teachers, certified evaluators observed the 
mathematics lessons of 50 randomly selected teachers. Ten additional teachers were contacted, 
but they did not participate in the observations for various reasons (e.g., not currently teaching 
mathematics, on personal leave, or too busy). As March of 1998, the ACME project records 
indicated that the number of ACME professional development hours that the 50 observed 
teachers had participated in ranged from 0 to 128 hours with a mean of 33.7 hours. Twenty-five 
observed teachers were considered untrained (i.e., had participated in less than 20 hours of 
ACME professional development), and 25 observed teachers were trained (i.e., had participated 
in 20 hours or more). The observation rating for one untrained teacher was omitted from the 
statistical analyses because his background was not representative of that group; he had a 
doctoral degree in mathematics education with a specialization in constructivist pedagogy. Most 
lessons sampled (86%) were in elementary classrooms, and the mode was fourth grade 
classrooms (n=l 3). 

To conduct 50 classroom observations, district evaluation and ACME project staff were 
trained and certified to rate classroom observations reliably using the HRI Classroom 
Observation Protocol (HRI, 1999b; see Appendix). Observers rated lessons on a 5-point ordinal 
scale, which ranged from ineffective to exemplary instruction on the basis of current standards for 
mathematics education. The purpose of sampling 50 mathematics classrooms was to gather data 
that were more representative than the 10 required observations conducted the previous year. 

Professional Development Observations 

The lead evaluator observed 1 1 ACME professional development sessions and formally 
rated five sessions with the HRI Professional Development Observation Protocol (HRI, 1999c; 
see Appendix). Similar to the classroom observation protocol, observers rated sessions on a 5- 
point ordinal scale, which ranged from ineffective to exemplary professional development 
facilitation. Observations lasted one to two hours. 

The observations covered a variety of ACME professional development activities. The 
five rated observations included the following: (a) a fourth grade and a seventh grade follow-up 
sessions during the school year; (b) a third grade summer institute session in which a classroom 
teacher modeled a lesson from the Investigations instructional materials; (c) a campus support 
visit made by a professional development facilitator; and (d) a seventh and eighth grade summer 
institute facilitated by an experienced Michigan trainer who had piloted CMP in her classroom. 
The informal observations of ACME professional development included the following: (a) a fifth 
grade follow-up session during the school year; (b) a “sneak preview” during the school year to 
provide information about ACME professional development for second and third grade teachers 
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who would begin participating in the summer of 1999; (c) a summer institute session on peer 
coaching with pilot school teachers; and (d) sessions with the teacher cadre, a group of trained 
teachers that ACME staff recognized as motivated to improve their own standards-based 
mathematics instruction and as leaders in reform. 

These sessions were selected because they represented: (a) the foundation of ACME 
professional development (i.e., week-long summer institutes and follow-up days during the 
school year); (b) teachers at all phases of participation in ACME professional development (i.e., 
from fifth grade teachers who initiated the project in the summer of 1997 to the most recently 
added cohort of third and eighth grade teachers); (c) elementary as well as middle school 
sessions; and (d) a variety of support services (e.g., on site support and peer coaching). 

Additional Sources 

Additional sources of information include interviews with district and ACME project 
staff, observations of district and project meetings, district and state mathematics curriculum 
documents, professional development materials, brochures, letters, and newsletters. 
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Results and Discussion 

ACME Professional Development Facilitators 

Five Categories of Facilitators 

Core ACME Team Facilitators 

In the second year of the project, ACME professional development facilitators fit into five 
categories. Five ACME mathematics specialists and one administrator made up the core of the 
ACME team. These facilitators were paid through the NSF grant, and provided the bulk of 
support to teachers and direction for the initiative. All ACME team members were former 
classroom teachers who were leaders in standards-based curriculum and instruction on their 
campuses, and some members had held administrative positions. . Many ACME team members 
had previously provided professional development about standards-based mathematics 
curriculum and instruction to other teachers on their campuses and for district, state, and national 
organizations. 

District Facilitators 

A second category of professional development facilitators were three mathematics 
specialists and one administrator hired by the district whose main responsibilities were to enact 
district policies and practices. These district staff supported the initiative by working with 
teachers and principals on campuses, by observing the day to day realities of implementing the 
curriculum resources, and by giving feedback to the ACME team. During the summer when 
work on campuses decreased, the district mathematics specialists and administrator helped 
provide ACME professional development. The AISD Mathematics Supervisor lead both the 
ACME team and the district mathematics staff. An additional facilitator served as a liaison 
between AISD and the Dana Center at the University of Texas. 

Middle School Facilitators 

Another category of professional development facilitators provided summer institutes to 
middle school mathematics teachers. Each year of the ACME project, ACME staff has hired a 
group of facilitators from Michigan schools who piloted CMP in their classrooms. These 
facilitators have extensive experience providing professional development to help teachers across 
the nation implement standards-based curriculum resources of CMP. The core team of ACME 
facilitators, on the other hand, provides follow-up professional development during the school 
year to all middle school mathematics teachers. 

Teacher Cadre Facilitators 

Two other categories of ACME professional development facilitators center on advancing 
teacher leadership in standards-based mathematics education throughout the district. In the 
second year of the ACME project, ACME staff hired a consultant with Marilyn Bums Education 
Associates to facilitate several sessions for a cadre of 30 elementary and middle school teachers 
who were highly motivated to implement standards-based curriculum resources in their 
classrooms and expressed deep understanding of standards-based pedagogy. These sessions 
focused on topics such as developing one’s own professional development project, promoting 
student discourse in the classroom, and cognitive coaching. From this teacher cadre, ACME staff 
selected another group of professional development facilitators. In summer institutes, elementary 
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classroom teachers who had participated in the teacher cadre professional development joined 
facilitators of the core ACME team and modeled lessons from Investigations for participants. 

Roles of ACME Team Members 

As ACME professional development facilitators, ACME team members were leaders in 
the district’s initiative to improve mathematics education with standards-based mathematics, 
both on and off campus. The symbolic move of leaving the classroom and being hired by the 
district’s Mathematics Department communicated their leadership roles. The demands placed on 
ACME staff and their own personal motivation to reach teachers and enact change in 
mathematics education bolstered their roles as leaders of the initiative and the district’s 
mathematics program. 

Professional Development Responsibilities 

The core ACME team provided most of the ACME professional development to teachers 
and other district staff at the district’s Professional Development Academy and on campuses. 
They designed ACME professional development activities and adapted pieces from other 
workshops and sources. These pieces centered on state and national standards for mathematics 
instruction, mathematics content, pedagogy, and the implementation of ACME curriculum 
resources. They developed other resources for teachers, such as assessment tools and 
accompanying professional development pieces, and generated documents to support teachers’ 
implementation of the ACME curriculum resources, such as charts that showed correlations 
between the investigations in the resources with the state standards (TEKS). ACME facilitators 
also “haul materials... and organize manipulatives,” as one team member described; the job was 
“labor intensive.” Additionally, through ACME professional development, ACME team 
members developed collegial relationships with district teachers and principals. For example, 
one ACME facilitator received a call on the thirteenth day of school from a teacher who had 
participated in a recent summer institute reporting with excitement that “her kids were finally 
learning math and were talking more about math.” Thus, ACME professional development 
facilitators provided teachers and district staff with material resources and one-on-one support 
for implementation. 

Campus Support 

In the second year of the ACME project, several ACME facilitators also supported 
teachers on campuses, especially at pilot schools. About twice a month, ACME team members 
visited these campuses to model teach, meet with grade level teams, co-teach, and mentor 
teachers. As one facilitator explained, these encounters were opportunities to “trouble shoot, 
listen to them talk about what’s going right, a time for reflection.” Often these encounters 
entailed what one facilitator called “meaningful minutes,” or brief interactions to share 
information about curriculum and instruction. Brief encounters with colleagues are common for 
educators whose workdays are focused on children (Lieberman, Saxl, & Miles, 1988). 

In one observation of campus support, many of these encounters were like touching base 
with teachers and providing moral and professional support for implementation of standards- 
based mathematics resources. Some teachers dug deep into how children’s actions and thinkin g 
had changed in their classrooms since the implementation of standards-based curriculum 
resources. However, in one brief encounter, the evaluator observed an educator who seemed to 
nod his head to standards-based mathematics curriculum and instruction while declaring himself 
a traditional mathematics teacher. The facilitator appeared to overlook the dissension in his 
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words and perhaps missed an opportunity to explore why he chose traditional instructional 
strategies and to suggest alternative approaches. A drawback of these brief interactions is that 
opportunities to influence change, particularly among educators who are hesitant about 
implementing standards-based mathematics, may be lost. Influencing change requires complex, 
in-depth interactions. 

Over the course of the year, ACME team members tried to develop conversation 
strategies that would influence change by helping teachers improve their mathematics instruction 
in meaningful ways. One facilitator realized that visiting teachers’ classrooms to observe their 
instructional strategies was a threatening format and detracted from constructive professional, 
conversations. She decided to try teaching a mathematics lesson in a teachers’ classroom, having 
that teacher observe, and then discussing what the teacher saw the students doing and saying. 
The subsequent exchanges were about children’s thinking and the facilitator’s instructional 
strategies — not the teacher’s instruction — and consequently were meaningful. This example 
demonstrates how the ACME facilitators pushed their own practice to arrive at the same end goal 
for educators as they intend to offer children: a deeper understanding through negotiated meaning 
in a non-threatening context. 

Preparation of ACME Team Members 

Attending Conferences 

ACME project staff made several different efforts to improve their knowledge and skills 
as professional development facilitators and leaders of mathematics reform. To build a 
foundation of professional growth, ACME team members attended various conferences across 
the nation to gather information and learn about what other professionals were doing to improve 
mathematics education such as the annual meeting of the National Council of Staff Development 
(NCSD), the Technical Educational Research Center (TERC) leadership conference, and the 
NCTM annual conference. The team appeared to put into practice what they had learned at these 
meetings through collaboration. Typically, what one team member learned was shared with the 
others. For example, at NCSD, one team member learned about a professional development 
strategy for empowering participants: In this strategy, participants explain their goals for a 

workshop and the facilitator matches the day’s agenda and organizes “dialogue groups” around 
those goals. The team adapted this approach to their own work with teachers. Another team 
member attended a two-week seminar at Developing Mathematical Ideas (DMI) to deepen her 
mathematics content knowledge. A few weeks later, she held a professional development 
meeting with the ACME team and mathematics staff to share what she had learned. Thus, 
information that team members gathered from outside sources was routinely exchanged with 
other members to enhance the ACME project. This collaboration reinforced the team’s 
commitment to ensuring consistency across ACME professional development for teachers: All 
teachers benefited from the knowledge of all ACME facilitators. 

Bridging on Others ’ Expertise 

ACME team members also sought professional growth by bridging on the expertise of 
others. New team members were oriented to ACME professional development by observing 
seasoned team members facilitate sessions and working with other members on specific projects. 
Before conducting their own sessions, new members co-facilitated sessions with experienced 
ACME team members. Contacting professional development providers at other sites was 
another vehicle for improving skills. Two new ACME team members were preparing to provide 
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campus support to teachers for the upcoming academic year. Not only did they consult with the 
team members who had provided this support the previous year, but they also met with a 
mathematics specialist from a New York City school district to discuss the strategies for campus 
support that she had developed. Thus, bridging on the expertise and skills of others was common 
practice for ACME team members. 

Improvements in the Quality of ACME Professional Development 

On the basis of professional development observations, the quality of ACME professional 
development generally improved as the providers honed their skills. Overall, the average rating 
of professional development observations for this year was higher than the average overall rating 
of the baseline year of the ACME project (Batchelder, 1998). In the baseline year, the average 
overall rating of ACME professional development on the HRI protocol was 3.6, which indicated 
the beginning stages of effective professional development in which sessions are somewhat 
limited in likelihood to enhance the capacity of teachers to provide high quality mathematics 
education (HRI, 1999c). In the present year, the average rating of ACME professional 
development was 4.1, which 'mdicdAed accomplished, effective professional development, which 
should affect the capacity of most participants to provide high quality mathematics education 
(HRI, 1999c). In particular, the delivery of mathematics content improved from a rating of 3.4 in 
the baseline year to 4.0 in the current year of the ACME project. However, these improvements 
may be artifacts of differences in the two samples: In the baseline year, all ACME team 

facilitators and district mathematics specialists were observed, whereas in the current year, a 
small number of facilitators were observed in a variety of settings. 

In the span of one year, ACME team members grew in their capacity to facilitate 
professional development at different rates. In one observation, for example, a facilitator 
appeared more relaxed and confident than she had been earlier. This confidence was reflected in 
her interactions with participants. She appeared more responsive to teachers than before such 
that she let participants do the talking in a large group discussion, listened, and summed up their 
ideas accurately. This approach contrasted with her performance the previous year when she lead 
the discussion and provided most of the ideas. Another facilitator softened her interactions with 
participants, but at times appeared not to address teachers as peers. Thus, the degree of 
improvement varied across ACME professional development facilitators. 

Support for the ACME Team 

Sources of support for the core group of ACME professional development facilitators 
included the ACME team itself, its leadership, and other district administrators. 

Team Spirit 

An outstanding characteristic of the core group of ACME professional development 
facilitators in the second year of the ACME project was team spirit. As one facilitator explained, 
“I couldn’t do it without the team... I collaborate with team members on everything; I don’t have 
to do anything by myself.” Although members brought different skills and knowledge to the 
ACME project, it was the process of communicating and sharing ideas as a team that promoted 
professional growth. One facilitator considered her most important professional development 
experience to be “meeting with the team, discussions, talking about making [professional 
development for teachers] better.” In the summer of 1999, team members talked at lunch 
informally about the professional development sessions they were facilitating. One teacher cadre 
member who worked with the ACME team said that she never realized how much thought. 




10 



18 



98.14 



ACME, 1998-99 Annual Report 



effort, and concern went into planning ACME professional development. Thus, the team 
members adopted the constructivist principles of collaboration and communication as means of 
continually improving ACME professional development. This collaborative effort provided what 
some team members considered their primary source of support and professional development. 

Integral to the ACME team spirit was the vision for reforming mathematics education in 
the district. Team members generally agreed that the purpose of the initiative centered around 
implementing standards-based mathematics curriculum resources in classrooms and having an 
impact on teachers districtwide. They agreed that ACME professional development should be 
ongoing and should provide opportunities for teachers to improve their mathematics content 
knowledge and to adopt standards-based pedagogy. One team member emphasized children’s 
experiences more than that of teachers. She stated that the ACME project was “about 
meaningful mathematics for kids, mathematics that is non-threatening and useful for kids.” 
Another team member took it a step further and asserted that the “overall goal was to change the 
way we’ve thought about teaching mathematics, to change the way we’ve thought about the 
culture and structure of teaching;... to dislodge the thinking, it’s not an individual effort; it’s a 
group effort of schools and the district.” Pulled together, these ideas suggest a plan for 
mathematics reform and seemed to fuel the team’s concerted efforts to support teachers. 

Team Leadership 

The ACME project leader was another primary support for ACME team members, and 
infused the team with a drive to constantly improve the quality of their work. The level of 
collaboration that ACME team members had achieved by the summer of 1999 was related in part 
to the ACME team lead’s efforts to learn from the highly experienced group of professional 
development facilitators from Michigan. At the end of the ACME project’s first year summer of 
1998, the ACME leader appeared to be concerned about the quality of professional development 
that the ACME team provided and sought ways to improve their skills and strategies for working 
together. That summer, when the Michigan group was facilitating professional development for 
AISD middle school mathematics teachers, the ACME leader observed how they worked 
individually as facilitators and as a team. At the end of each day, the Michigan facilitators 
discussed their work by problem-solving and trying to figure out how to improve the sessions to 
meet teachers’ needs. Although the ACME team had established a system of reviewing teacher 
evaluation forms concerning ACME professional development, they generally processed the 
feedback as judgments or complaints rather than as problems to solve. The ACME director 
returned to her team in the fall to refocus the conversations about professional development and 
to improve their work by listening to what teachers were saying. This example also demonstrates 
the adaptability and the climate of improvement through learning of the ACME team’s 
leadership. 

Other Administrative Support 

In general, ACME professional development facilitators reported that in addition to team 
work and the ACME team leadership ainple funds supported their efforts to promote standards- 
based mathematics in the district; the ACME facilitators “want for nothing.” Additionally, 
within in the district, they had a “network of people to call,” such as specialists in staff 
development. Central office support affected the work of the team lead directly and was filtered 
down through the ACME team lead. Support from campus administrators, however, was mixed. 
Whereas one principal sought help from ACME team members to put a plan in place to support 
teachers who were implementing standards-based resources, another questioned the ACME team. 
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curriculum resources, and standards-based instructional strategies often. ACME team leaders, on 
the other hand, who provided professional development directly to campus administrators, 
reported that many were enthusiastic and wanted to learn more about what they could do to 
support the initiative and teachers on their campuses. Thus, in the second year of the ACME 
project, support for ACME team members was strong, but not complete on all fronts. 

Changes in the ACME Team 

The ACME team’s drive to constantly grow professionally has resulted in changes in how 
the team works with teachers, other district staff, and the community. In the first year of the 
ACME project, the team had adopted the metaphor of a “breastplate” to protect themselves from 
the onslaught of attacks from people who were opposed to the ACME initiative or reluctant to 
participate. In its second year, ACME team members took off the breastplate because they no 
longer needed it. They had softened from a defensive posture to a receptive, open posture; they 
listened to teachers more and talked at them less than before. Teachers themselves remarked 
these changes in the ACME team’s attitude. One teacher stated, “In the beginning everyone was 
defensive; they [facilitators] put energy into it, at latter [sessions], they acknowledged 
weaknesses, listened to your thoughts; that was great...; the new attitude of ACME staff is 
wonderful.” 

This evolution in the ACME team’s approach to teachers reflects the integration of 
another constructivist principle into ACME professional development: The ACME team has 
moved from a unidirectional to a bidirectional approach in their work with teachers. As one 
ACME team member explained, “We’ve evolved in our goal, the team one year ago said, ‘This is 
what teachers need,’ so now we’re asking, ‘What do teachers want? What is important to 
them?”’ With this change in approach, the team has devised a separate professional development 
session for special education teachers, for example, which was hailed as a unique opportunity 
because their needs for implementing standards-based mathematics were not met in other 
sessions. 

Another example of the team’s responsiveness to teachers was the development of a 
packet for kindergarten through eighth grade teachers that included (a) recommended unit 
sequence for the implementation of AISD curriculum resources; (b) charts that correlate the 
TEKS state standards for mathematics with the AISD adopted curriculum resources; and (c) lists 
of the mathematical content emphasis of those resources with strategies for differentiation for 
gifted and talented, special education, and bilingual/ESL students. Another example of how the 
team responds to the needs of teachers was to gamer financial support from the district to 
alleviate the time elementary teachers spent making copies of student sheets for investigations. 
In the fall of 1999, all teachers will receive shrink-wrapped stacks of student sheets for their 
classes. 

Teachers’ Experiences in ACME Professional Development 

Many teachers who participated in ACME professional development had positive 
attitudes about its quality. Among the teachers surveyed, 57% rated the overall quality of ACME 
professional development as “good,” “very good,” or “excellent;” 33% rated it “fair,” and 10% 
rated it “poor” or “very poor.” 
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Addressing Teachers’ Needs 

In the second year of the ACME project, teachers’ experiences in ACME professional 
development reflected the ACME team’s focus on listening to and addressing the needs that 
teachers expressed. As one teacher explained, “The people are trying to gear next time so that 
it’s better for you, makes you know you’re being heard.” In several observations, the design of 
ACME professional development catered to the needs of teachers. The facilitator of one fifth 
grade follow-up session organized the “dialogue groups” previously mentioned around goals 
teachers set at the beginning of the session. The topics teachers selected included applications of 
computer technology to investigations, assessment strategies that teachers generated, and 
differentiation for special education students. In another observation, the design of ACME 
professional development directly addressed middle school teachers’ request to learn how to plan 
CMP lessons. The teachers devised a standard and applied it to the planning of one unit. 
Additionally, the ACME team brought in classroom teachers to model lessons, because many 
elementary teachers said that they wanted sessions on “Here’s how to do lessons.” Thus, these 
examples illustrate a few of the adaptations that the ACME team made to professional 
development to accommodate teachers. 

Opportunities to Share with Colleagues 

In a recent case study of nine AISD schools (Batchelder & Christian, 1999), teachers cited 
opportunities to share ideas about implementing standards-based mathematics curriculum 
resources with colleagues as one of the most beneficial experiences of ACME professional 
development. As one teacher stated, “It gives you opportunities to talk with colleagues on your 
campus and others; it gives you a good impression of what’s going on.” In one of the dialogue 
groups previously mentioned, special education teachers exchanged ideas about pedagogy and 
applications of standards-based curriculum resources with their students. Some of the ideas that 
these teachers expressed during the professional development observation follow; 

• Some special education students fit well with Investigations, some 
students do not. 

• In cooperative groups, kids tutor kids. Teaching can help them with 
individual work. 

• Because the curriculum involves student-directed rather than teacher- 
directed learning, special education students are successful. 

• Investigations is difficult to read and convoluted. 

• Fifth grade special education kids can do third grade exercises at third 
and fourth grade level, then through inclusion students understand the 
mathematics in the fifth grade class. 

This example illustrates how ACME professional development provided teachers opportunities 
to discuss their thoughts about using the curriculum resources with their students and specific 
strategies that they used. Noteworthy is the willingness of teachers to express praise as well as 
discontentment for the resources and instructional strategies. 

In the professional development observation of middle school teachers planning a CMP 
lesson, participants dug deep into mathematics content and pedagogy. Many teachers from the 
same campuses and from campuses across the district had lively discussions and collaborated 
actively in planning. In this session, the observed teachers talked about possible directions to 
take the mathematics and got excited about the ideas of other teachers. They proposed additional 
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problems with similar content that students could solve. The discussion of mathematics content 
flowed naturally into pedagogy. Teachers discussed what they thought about putting right and 
wrong answers on the board and how the practice had worked with their students. They also 
suggested follow-up questions to ask for that lesson. A majority of the teachers in the session 
appeared to learn a great deal about how to use the curriculum resources with their students. 

Opportunities to Share with ACME Facilitators 

Additionally, teachers had opportunities to share ideas about mathematics instruction 
with facilitators in ACME professional development. In the observation of middle school 
follow-up just described, teachers benefited from the facilitator’s probing questions and 
knowledge of the curriculum resources. While discussing the lesson with a colleague and the 
facilitator, one teacher brought up the point that a probability is always between zero and one and 
gave the following example, “If you throw a ball up, it won’t stay in the air.” The facilitator 
asked the participant, “Is that something that you’ll need to pull out because it’s not in the actual 
investigation?” This question pushed the teacher to consider how he could present the concept to 
students, for example, with a scale from zero to one. This dialogue not only pushed thinking of 
this educator further, but it also gave other teachers ideas of instructional strategies to pursue 
with students. 

In another observation of professional development of fourth grade follow-up, an ACME 
facilitator discussed ways to help children multiply by clustering with a small group of 
participants. The discussion focused on how children would approach the problem, particularly 
special education students. One teacher said, “One child in my class still has trouble reasoning.” 
The facilitator suggested an alternative way to solve the problem and said, “Would he get it this 
way? If he’s not getting it this way, he’s building on these other concepts here.” Another teacher 
at the table said, “It’s easy if you’re teaching kids who have good number concept.” The 
facilitator suggested, “Well can he count by tens.” In this example, teachers were unsure about 
helping some children successful solve problems with clustering. The facilitator tried to help the 
participants think about how the strategy could be useful by suggesting different approaches, and 
by connecting to approaches to the problem with which the student may be familiar. 

ACME professional development also established a venue where teachers could learn 
about district mathematics policy and practices directly from ACME facilitators, who were 
central office staff, and voice concerns. For example, when the district adopted two mathematics 
curriculum resources in the spring of 1998, ACME facilitators clarified district policy: 
Investigations and CMP were the main curriculum resources that teachers would use and the 
supplemental texts were adopted to fill in state standards (TEKS) that the other resources did not 
cover. One teacher asked why the books were selected, and an ACME facilitator who had been 
on the selection committee explained the rubric used to evaluate resources and the process of 
documenting their ratings. A few teachers aired frustrations about having two adopted resources 
and their difficulties using Investigations. Other teachers articulately promoted standards-based 
resources. Thus, the interaction between teachers and ACME professional development 
facilitators went beyond learning about instructional strategies and materials, which yielded 
positive results. On the basis of open discussions like this one, the ACME team decided to 
develop materials such as charts that correlate the TEKS standards with all AISD adopted 
curriculum resources. 
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Teachers’ Engagement in ACME Professional Development 

Incomplete Teacher Engagement in Activities 

Although ACME professional development provided teachers with opportunities to 
explore standards-based curriculum resources and instruction, in the second year of the ACME 
project, as in the baseline year, participation was not 100%. Among the teachers surveyed who 
had attended ACME professional development, 68% reported that they had participated in an 
ACME discussion group during the academic year at least once, and 44% had participated in a 
discussion group three or more times. Because all ACME professional development involved 
small and large group discussions, every teacher who attended sessions would have been invited 
to participate in discussion groups with other teachers. These percentages correspond to 
professional development observations. In more than one observed session, one-fourth to one- 
third of the tables had teachers who were not actively involved in ACME professional 
development activities. In the observation of the middle school session described previously, for 
example, nine out of 12 participants actively engaged in the activities. Some teachers attended 
but appeared to be punching the clock. During various observations, one teacher was observed 
reading a newspaper, another was making paper airplanes, and still another was looking at a 
history text. 

Certainly, in any initiative that asks some to change their current practice and intends 
systemwide participation, disgruntled participants will surface. One AISD teacher stated that she 
disliked the amount of time spent in professional development and said, “It’s more than what’s 
necessary, the 8 hour span. They come up with 8 hours of activities, but the material could be 
covered in half the time. My gut feeling is that that’s the number of hours with the grant, but a 
lot of them are not necessary.” This teacher did not want to invest the time in ACME 
professional development and did not find value in many of the activities. Some teachers were 
disgruntled or reluctant to implement standards-based mathematics in their classrooms (see 
examples in Batchelder & Christian, 1999). It is also possible that some teachers do not 
participate in ACME group discussions because they may be timid about speaking in large 
groups, particularly about a topic new to them. 

Facilitators’ Adjustments to Incomplete Teacher Engagement in Activities 

In response to the objections of some teachers, the ACME team changed their approach to 
teacher participation from “required” to “expected.” (In the first year of the project, additionally 
the teachers’ union pointed out that attendance could not be required because participation was 
not specified in teachers’ contracts.) The ACME team members also reflected on what they 
could do to make teachers want to attend to ACME professional development. They decided to 
make explicit to teachers and administrators what participants would gain by participating, for 
example, in a “sneak preview” session for new participants. 

Although encouraging teachers to attend ACME professional development does not 
guarantee that teachers will engage in the activities. Flexibility and expectations are two 
strategies for engaging participants. Flexibility to engage in activities may open up the field and 
make more teachers feel welcome to participate. One facilitator introduced a summer institute 
with a sharing exercise in which people told stories about something that was unique about 
themselves and invited other people to build on the stories by pointing out commonalities 
between themselves and others. Rather than going around the room and insisting that everyone 
participate, she let those teachers share who wanted to and then flowed naturally into the next 
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activity. This approach took the pressure off those participants who did not want to share 
personal stories and maintained a welcoming atmosphere. 

However, flexibility to engage in activities should not preclude communicating the 
expectation that attendees will participate in the day’s events. In professional development 
observations, some ACME facilitators appeared at times to ignore or overlook tables of teachers 
who did not actively engage in the activities. Perhaps a question or redirection would have 
pushed teachers to engage actively. Facilitators could also make on the spot decisions to pair 
inactive participants with active ones. In one observation, for example, half of the small groups 
were more active than other groups. When the facilitator paired small groups into larger groups, 
she matched groups that had highly active participants with other groups that also had active 
participants. If she had organized the groups differently, that is, active with inactive participants, 
the synergism may have raised the overall level of engagement in the activities. In sum, ACME 
facilitators could find other strategies for communicating the expectation that teachers will 
engage in activities within the session. 

Other ACME Professional Development 

Two professional development formats, campus support and teacher cadre, were offered 
to a small proportion of AISD teachers. Teachers who participated highly valued these 
opportunities for professional growth. 

Campus Support 

One ACME professional development experience that teachers raved about was campus 
support. In the case study of nine AISD schools (Batchelder & Christian, 1999), many teachers 
at pilot schools where ACME facilitators provided campus support said they appreciated the help 
they received from support on campus. Teachers could call with a question about the curriculum 
or ask to be observed. One teacher at a pilot school praised campus support and said, “They’ve 
done a very good job, they’ ve . . .tried to help us iron out problems.” 

At another pilot school, the entire fourth grade team of teachers wrote and signed an open 
letter “to express their appreciation and professional regard for the services and leadership 
provided by” an ACME facilitator who provided support on their campus: 

We have enjoyed her perspective, depth of knowledge, honesty, and sense 
of humor. She has always been available to assist or find help, when 
requested, offered direction and suggestion, as well as guided us in 
preparation for using “Investigations” in our classes and TAAS [Texas 
Academic Assessment Test] testing.... Because of her sensitivity, we feel 
we’ve been listened to and respected as professionals. We look forward to 
her continued mentorship of us as we explore new math horizons with our 
students. 

In this letter, the teachers asserted their appreciation for the one-on-one mentoring they received 
in mathematics instruction from on campus support. These words indicate that as a team they 
formed collegial relationships with the ACME facilitator and benefited from her expertise in 
mathematics and professional direction. 

It is important to note that teachers seem to appreciate campus support in part because it 
may pull them out of the isolation of classroom teaching. However, as the case study of nine 
schools demonstrated, pilot schools, which received campus support from ACME staff, were not 
further along in implementation than were other schools; teachers displayed a range of 
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competence in standards-based instruction at all schools observed (Batchelder & Christian, 
1999). Perhaps, teachers with expertise in standards-based mathematics on campuses could 
provide leadership and mentor their colleagues in reflective practice to improve instruction 
optimally. 

Teacher Cadre Professional Development 

As previously described (see “ACME Professional Development Facilitators, Five 
Categories of Facilitators”), teacher cadre meetings provided advanced professional development 
to some teachers who were skilled in standards-based mathematics instruction. One teacher 
whose passion was standards-based mathematics stated that the teacher cadre meetings were 
“terrific.” She considered the meetings to be the most helpful aspect of ACME professional 
development, and said: 

It offers ways to get in depth, hear different points of view and 
perspectives.... It brings you back to class rejuvenated. It’s given me 
other ideas and ways to talk about things. It’s O.K. Other teachers have 
the same concerns and successes, there’s commonality; you’re not 
alone.... There’s support there, you’re more able to take risks, when 
you’ve got people behind you. 

The meetings were a venue where skilled teachers could meet and form professional 
relationships with other teachers with similar competence in standards-based mathematics 
instruction. 

Shortcoming of Other ACME Professional Development 

One shortcoming of campus support and teacher cadre professional development merits 
consideration. Both of these alternative forms of professional development touch a small number 
of teachers. In the second year of the ACME project, only eight pilot elementary schools 
received campus support, and only 30 teachers were members of the teacher cadre. Although the 
ACME team did not have the staff resources to develop relationships with every elementary and 
middle school campus in the district, the district did. As the case study of nine schools revealed, 
teachers’ expertise in standards-based mathematics instruction is typically underutilized on 
campuses (Batchelder & Christian, 1999). Campus administrators and teachers themselves could 
organize campus mentorships to help less skilled teachers build their capacity through the use of 
experts on campus. 

Effects on Teachers’ Knowledge and Siolls 

Classroom Observations of Mathematics Instruction 

Definition of Rating Scale 

Mathematics lessons were rated using the Classroom Observation Protocol (HRI, 1999b) 
on an 8-point scale ranging from ineffective to effective standards-based instruction. Level 1 
refers to ineffective instruction in which there is little evidence of student engagement with 
mathematical ideas. Level 1 has two subcategories: Level lA involves passive learning in which 
the students receive knowledge from the teacher or text and Level IB is activity for activity’s 
sake in which hands-on lessons lack purpose or content. Level 2 describes instruction with 
elements of effectiveness that has substantial problems in the design, implementation or content 
of the lesson, and is limited in the likelihood to enhance children’s mathematical knowledge. 
Level 3 refers to beginning stages of effective instruction characterized by a few elements of 
effectiveness that frequently engage children in mathematical concepts, but has some 
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weaknesses. Level 3 is divided into three ordinal subcategories of low, solid, and high. Level 4 
reflects effective instruction that is engaging for most students, whereas level 5 describes 
exemplary instruction that engages all of the students most of the time and represents the art 
more than the craft of teaching. 

Classroom Observations of Trained and Untrained Teachers 

Figure 1 presents frequency distributions of the 50 classroom observation ratings for 
trained (i.e., 20 hours or more of ACME professional development) and untrained (i.e., less than 
20 hours of ACME professional development) teachers. The frequencies indicated that ACME 
professional development was related to a high number of lessons rated effective. The lessons of 
15 trained teachers were rated as beginning stages of effective instruction at levels 3 low, solid, 
and high, whereas the lessons of 10 untrained teachers were rated level 3. Additionally, the 
lessons of no trained teachers were rated as ineffective instruction at levels lA or IB, but the 
lessons of three untrained teachers were rated levels lA or IB. About the same number of 
trained and untrained teachers were rated at levels 4 and 5. 

Figure 1. Frequency Distributions of Classroom Observation Ratings 
by Teachers’ Training in 1999 



Untrained Trained 

Teachers' Training 





Examination of the histogram reveals that the distribution of the ratings of the lessons of 
untrained teachers was bimodal centered around levels 2 to 3 low and level 4 and was more 
variable than the distribution of ratings of the lessons of trained teachers. A test of homogeneity 
of variance was not significant (Levene’s statistic {SDi =1.95, SD 2 =1.59) = .85), which attests to 
the differences in the variability of these two distributions. Thus, participation in ACME 
professional development made the instruction of teachers similar and concentrated at the 
different subcategories of level 3 (i.e., low, solid, and high). 

Examination of the central tendency of the two groups revealed a trend for the rating of 
the lessons of trained teachers (mean rank = 27.74) to be higher than the rating of the lessons of 
untrained teachers (mean rank = 22.15, Mann-Whitney U = 231.5, p < .10). Thus, ACME 
professional development tends to influence inquiry-based instruction among AISD mathematics 
teachers that is more effective than the instruction of teachers who do not participate. 

However, the number of hours that teachers had participated in ACME professional 
development was not correlated with the rating of their lessons on this 8-point scale. This 
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finding may be due to the fact that teachers begin ACME professional development with 
different skill levels in standards-based mathematics instruction. The eight untrained teachers 
whose lessons were rated at level 3 high or above suggests that a number of teachers who have 
not participated in ACME professional development have those skills, which are goals of the 
ACME project. This finding suggests that the district has a pool of resources to support 
standards-based mathematics education that may not be fully tapped. 

Insights from Observers 

The certified evaluators outlined several insights about standards-based instruction on 
AISD campuses from observing the lessons. Mathematics content knowledge was a key factor 
that distinguished between a lesson that lacked effectiveness (i.e., levels 2 or 3 low) and one that 
did not (level 3 high or above). If teachers had limited content knowledge, they were unable to 
explore the mathematics of the lesson deeply, sometimes confused students, and missed 
opportunities to help children expand their understandings. Classroom culture also distinguished 
between effective and ineffective lessons. In these lessons, teachers were uncomfortable with 
standards-based pedagogy, presented teacher-driven lessons, and cut off student exploration of 
the mathematical concepts. 

The characteristics of lessons that were rated accomplished or exemplary instruction (i.e., 
levels 4 and 5) centered on knowledge and skills in standards-based pedagogy as well as refined 
classroom culture with deep exploration of mathematical concepts. These teachers were facile 
with the mathematical content knowledge and asked students probing questions. These 
educators got involved in the learning of almost every child in the classroom, and most children 
seemed to move from one level of understanding to another. These teachers appeared to have 
positive regard for all of the students in the classroom and high expectations that the students 
would understand the mathematics behind the lesson. 

Deepening Teachers’ Understanding of Mathematics Content 

Activities for Adult Learners 

One of the main concerns of the ACME team at the end of the baseline year was how to 
help teachers gain mathematics content knowledge through ACME professional development. In 
the summer institutes of 1999, the ACME team presented teachers with engaging problems to 
solve that would allow them to deeply explore mathematical concepts and reflect on their 
experiences as learners. The ACME facilitators designed activities that, as one team member 
said, would “put teachers in math situations and make them push it further.” For example, 
ACME facilitators asked second and third grade teachers in small groups to skip count by 21s, to 
predict what number would be at the end by looking for patterns, and ultimately to write an 
algebra statement. Thus, by designing activities in which teachers explore mathematics at levels 
higher than they teach, the ACME team helped teachers deepen their mathematics content 
knowledge. Rather than lecturing about mathematics content, the ACME team applied 
constructivist principles in their work by helping teachers learn mathematics content through 
inquiry. 

The ACME team also helped teachers deepen their mathematics content knowledge by 
examining children’s mathematical thinking and problem solving. They had teachers develop 
their own computational strategies because many learned to solve mathematics problems; by 
following the steps of algorithms. These exercises allowed teachers to look at mathematical 
concepts as their students might. In other activities, teachers analyzed mathematical concepts in 
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student work, which allowed educators to apply and develop their understanding of mathematics. 
For example, the participants examined student work and developed assessment strategies that 
would ensure interrater reliability. Additionally, the ACME team designed activities in which 
teachers explored mathematics content knowledge through infusion. The ACME facilitators 
engaged teachers in activities that addressed computation, measurement, algebra, geometry and 
other topics as they became familiar with the instructional materials. 

No inaccuracies in the presentation of mathematics content were observed during 
professional development activities. Among teachers who were engaged in ACME professional 
development activities, many applied themselves and struggled with mathematics topics of which 
they did not have a complex understanding. 

Mathematics Content Knowledge Gained 

Although the ACME team tailored professional development activities to the needs of 
adult learners, the approach to teachers’ mathematics content knowledge may have a smaller 
impact than tO' other areas of standards-based instruction. Teachers who were asked about the 
impact of ACME professional development on their teaching rarely talked about mathematics 
content knowledge, even when explicitly asked about content. It is as if they did not have the 
words to describe the mathematical concepts underlying the curriculum resources. When asked 
about changes in the mathematics content of his teaching, one educator stated, “It’s hard to 
say.... I haven’t thought of activities outside the box.” Another teacher when asked about 
content referred to changes in her pedagogy; rather than teaching children basic math facts, she 
said that “[Investigations] has moved me more towards problem-solving and everyday math.” It 
may be that the lack of commentary on mathematics content reflects the fact that the random 
sample of interviews included only elementary teachers, who typically do not have a background 
in mathematics. Middle school teachers, in contrast, who specialize in specific content areas, 
were observed discussing complex mathematical concepts (see previous discussion in “Teachers’ 
Experiences in ACME Professional Development”). 

Of the teachers surveyed who had participated in ACME professional development, 39% 
reported that their participation increased their mathematics content knowledge “somewhat” or 
“to a great extent,” whereas 44% reported that participation in ACME increased their 
understanding of how children think about/leam mathematics and 51% reported that participation 
increased their ability to implement high quality mathematics instructional materials. Similarly, 
among teachers surveyed in 1999, there was a trend for more teachers to report feeling more 
prepared to teach different areas of mathematics content (e.g., measurement, algebra, and 
geometry) than in the previous year, but the results for increases in pedagogy were more 
powerful. 

The ACME project has not established a means for evaluating the depth of mathematics 
content knowledge that teachers gain through ACME professional development. The ACME 
facilitators themselves expressed uncertainty about how well teachers learned mathematics 
content. One facilitator said that in conducting five weeks of summer institutes she observed a 
few “Ah-has” per week from teachers about content in the large group (although this measure 
does not document realizations that teachers keep to themselves). Another facilitator stated that 
she found out about teachers’ understanding of mathematics content through conversations with 
teachers about ‘what kids understand.’ This facilitator stated that often she did not hear much 
beyond awareness of content; deep conceptual understanding was rare. Thus, the ACME team 
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has observed some improvements in teachers’ mathematics content knowledge, although these 
gains may not be deep dr pervasive districtwide. 

Evidence from classroom observations suggests that trained teachers who participated in 
20 hours or more of ACME professional development become more effective in standards-based 
instruction of which mathematics content is a part. Although many teachers presented the 
content of Investigations and CAfP lessons accurately, some trained teachers were observed 
mixing up the concepts of surface area and volume and confusing her students. Other teachers 
treated content at a superficial level or misinterpreted students’ questions because the educators 
lacked content knowledge. Assessment of the mathematics content knowledge that teachers gain 
in ACME professional development may suggest improvements to make in activities. 

Improving Teachers’ Knowledge of Pedagogy 

Conversations about Pedagogy and Student Work 

One area of standards-based instruction that the ACME team tackled and improved in the 
past year was pedagogy and helping teachers to develop an understanding of students’ 
mathematical thinking. The team attacked these teaching strategies from several different angles. 
As facilitators, they modeled pedagogy and asked teachers questions such as “How did I facilitate 
this activity to help you understand the mathematics?” Although in the previous summer the 
team had included pieces on open-ended questioning strategies, in the Summer of 1999 
facilitators raised the bar and stimulated professional conversations about these strategies through 
a book study of Beyond Arithmetic (1995) and examining Bloom’s taxonomy as related to the 
state standards TEKS. Additionally, the ACME team made videos of AISD teachers skilled in 
standards-based pedagogy, screened the videos in ACME professional development, and held 
group discussions of the observed teaching strategies and students’ dialogue. ACME facilitators 
also had teachers share their own experiences in class as well as predict what students might do if 
they posed questions one way rather than another. 

By honing their approach to assessment strategies, the ACME team helped teachers 
improve their understanding of student’s mathematical thinking. The ACME team members 
designed a professional development piece to promote better reliability across teachers on the 
district’s performance assessment program. They focused on rating student work with rubrics on 
a developmental continuum of low, medium, and high and on the dimensions of mathematical 
thinking, communicating support for thinking, and mathematical skills. By examining teachers’ 
rating from previous ACME professional development, they noted a misunderstanding: Teachers 
considered students’ different representations of solutions as distinct strategies when actually 
children had presented just one strategy in several different ways. In ACME summer institutes, 
teachers learned to rate student work and participated in professional conversations about student 
mathematical understanding. In follow-up professional development during the academic year, 
teachers will bring their samples of student assessments and continue working on applying 
rubrics and developing. 

In ACME professional development, ACME facilitators explored with teachers the needs 
of diverse learners by discussing extensions and adaptations to activities. At teachers’ requests, 
the ACME team developed the planning tool of charts with extensions for gifted and talented, 
special education, and bilingual/ESL students. These charts covered every investigation of every 
book of Investigations and CMP. 
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Teachers' Knowledge of Standards-Based Pedagogy Gained 

The ACME team appears to have had an impact on some teachers’ knowledge of 
standards-based pedagogy. Teachers surveyed in the Spring of 1999 reported significantly higher 
levels of preparation to implement standards-based instructional strategies (e.g., have students 
participate in hands-on activities and work in cooperative groups) than did teachers in the 
previous year. Additionally, significantly more teachers surveyed reported that they used 
standards-based instructional strategies more often than did the previous year. However, these 
increases are based on teachers’ self-reports and were not confirmed by observation. 

The impact that the ACME project had on teachers’ knowledge of standards-based 
pedagogy also emerged in how they talked about changes in their pedagogy. Their words echoed 
the emphasis that the ACME team placed on solving problems with multiple strategies. One 
teacher said that standards-based instruction “allows for children to explain their thinking 
processes; it made me aware of how divergent different children’s thinking processes are.” 
Another teacher said that the most helpful aspect was that standards-based curriculum allowed 
“students to develop their own strategies and share them with others; there’s not just one way of 
doing a problem.” Another teacher noted the importance of questioning, and said that her 
teaching strategies in mathematics included “more problem-solving, inquiry; math is learning 
from questions.” One teacher stated that standards-based pedagogy helps reach diverse learners; 
“It’s shown me that there is a way to reach all children, irregardless of their level, with hands on, 
it reinforces my idealism that all children can learn.” 

Skills in Standards -Based Pedagogy in Classrooms 

Although ACME professional development provided opportunities for teachers to acquire 
knowledge of standards-based pedagogy, the activities are one step removed from what the 
initiative attempts to change, that is, what teachers do in classrooms. Classroom observations 
indicated that ACME professional development influenced change in instruction such that more 
trained teachers demonstrated effective instruction than untrained teachers. However, in the case 
study of nine schools, teachers demonstrated a range of skills in standards-based instruction 
(Batchelder & Christian, 1999). The finding that teachers were able to articulate principles of 
standards-based pedagogy does not necessarily imply that this knowledge transfers to 
pedagogical skills in the classroom. With campus resources such as experts in standards-based 
instruction, direct observation and feedback may help teachers apply this knowledge and hone 
their skills in standards-based pedagogy in their classrooms. 

Learning How to Use the Instructional Materials 

Availability of Materials 

When the ACME project began, all teachers received a set of the Investigations and CMP 
curriculum resources and schools received one kit of manipulatives for every two teachers. In 
response to teachers’ requests for their own set of manipulatives, the AISD Mathematics 
Department applied for and the AISD board of trustees approved funds for one kit per 
mathematics teacher in the 1999-2000 academic year. Also, in response to teachers’ needs, the 
board approved funds for copies of student sheets for every teacher’s class because educators had 
spent too much time making copies. 

Approaches to Instructional Materials in ACME Professional Development 

To help teachers become comfortable with and learn how to implement standards-based 
instructional materials, several different approaches have evolved. In the first year of the ACME 
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project, ACME team members designed a scavenger hunt to help teachers learn the components 
of the resources. The team has continued to launch ACME professional development for first- 
time participants this way. Throughout ACME professional development, participants were 
asked to engage in activities that exposed them to manipulatives, to play the games in the 
resources, and to explore the mathematics underlying the activities. In follow-up sessions, 
professional development activities focused on books that teachers were scheduled to use in the 
coming months. In these professional development pieces, ACME facilitators encouraged 
participants to discuss the materials one-on-one, in small groups, and in large groups, and to 
share experiences from their classrooms. These discussions included how to organize materials 
and classroom management. Most recently, the ACME team modified summer institutes by 
bringing in classroom teachers from the teacher cadre who were proficient in standards-based 
pedagogy to model lessons from Investigations. An observation of one of these sessions was 
convincing because the teacher was able to draw on her recent experiences using the materials in 
her classroom and describe what worked and what did not. 

Teachers’ Knowledge and Implementation of the Instructional Materials 

Most of what teachers appreciated about ACME professional development were the 
instructional materials and support for learning how to use them. Some teachers appeared to 
have a thorough knowledge of the contents of the materials. One teacher said, “I love the 
Investigations books, the layout, the different investigation materials needed, the teacher notes, 
the problems kids encounter, ways to reach kids, the activities, and manipulatives.” Another 
teacher appreciated the organization of the books, and said, “How it has been set up— ordering of 
how to teach in the books, letters for parents, assignments, student worksheets; there’s an order 
to it; versatility, it allows me to add things that I think my students need.” Other teachers did not 
appreciate the organization or the content of the materials. One teacher said, “Some of the 
choice times, some in different books don’t seem to be meaningful at all; they’re repetitive.” 

In general, AISD teachers who participated in ACME professional development became 
more comfortable with the instructional materials as they progressed through the program. In 
classroom observations, some teachers, especially those who had participated in two years of 
ACME professional development, appeared to be comfortable with the materials and 
demonstrate some effectiveness in their instruction. Other teachers, particularly those in their 
first year of implementation, appeared shaky and uncomfortable with the materials, sometimes 
reading from the book. 

Implementation of ACME Professional Development 

The ACME professional development model is grade-by-grade implementation for all 
AISD elementary and middle schools. Eight pilot elementary schools implemented the ACME 
project such that teachers at all grade levels participated in professional development 
simultaneously. The implementation of the design of the ACME project is on schedule; Most 
fifth and sixth grade teachers completed ACME professional development in the Spring of 1999, 
most fourth and seventh grade teachers completed it in the Spring of 2000, and most second, 
third, and eighth grade teachers completed it in the Spring of 2001. Kindergarten and first grade 
teachers will attend their first institute in the summer of 2000 and complete two years of 
professional development in the 2001-2002 academic year. 

Attendance at initial and follow-up institutes in the summer of 1999, however, was less 
than 100%. In AISD, each elementary grade level had approximately 250 teachers and each 
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middle school grade level had approximately 60 mathematics teachers. Approximately 20% of 
AISD elementary schools departmentalize subjects in the fourth and/or fifth grades so that half of 
the teachers in those schools do not teach mathematics. The attendance at the summer institutes 
of 1999 for second and third grade teachers was comparable to the attendance fourth grade 
teachers the previous summer; approximately 90% of the teachers who were expected to attended 
the first week of the 1999 summer institute and approximately 75% attended the second week 
(Batchelder, 1998). Similarly, 80% of eighth grade teachers attended the first summer institute 
of ACME professional development in 1999. As for fifth grade teachers in the previous year, 
attendance at follow-up summer institutes for fourth, sixth, and seventh grade teachers tapered 
off to about 60% of the expected attendance. 

On the basis of attendance, it appears that for the past two years not all teachers targeted 
were participating in ACME professional development. By the end of the summer of 1998, 
ACME staff figured out that many new teachers were hired before school started and after 
ACME summer institutes ended. As a result, they were unable to attend summer institutes until 
after they started teaching. The ACME team lead reported that high turnover in the district 
amounted each year to about 60 new hires at the elementary level and about 20 new hires at the 
middle school level. In addition, ACME staff noted that some teachers changed grade level 
positions often from one year to the next. In response to this transience, ACME staff decided to 
repeat summer institutes for grade levels for which the cycle through two years of ACME 
professional development had already been started or was complete. They also held brief one- 
day overviews about standards-based mathematics in the first month of school. Some established 
AISD teachers who had skipped the summer institute that they were targeted to attend also 
participated in the repeated summer institutes. For fourth, sixth, and seventh grade teachers, 
attendance at repeated institutes in 1999 was relatively high at 63% to 80% of the expected 60 
participants, but not for fifth grade teachers at 45% of the expected number. 
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Summary and Recommendations 

Strengths of ACME Professional Development 

• One key strength of ACME professional development is how the core ACME team 
members work together and their vision of high quality standards-based professional 
development. The incorporation of constructivist principles such as collaborating, 
bridging on the expertise of others, and pursuing professional growth solidifies 
continuous improvements in the quality of their work. 

• The abundance of collegial and financial resources provides invaluable support to the 
ACME team. 

• The strong ACME team leadership pushes the team members to improve their work by 
analyzing available information about problems and finding workable solutions. 

• The ACME facilitators make themselves vulnerable and listen to the needs of teachers. 
Many changes in the design of the ACME project that cater to teachers’ needs have come 
from this openness. 

• ACME professional development provides many rich opportunities to explore standards- 
based curriculum resources and instructional strategies for those teachers who actively 
engage in activities. 

Challenges of ACME Professional Development 

• The ACME team needs to find ways to improve teachers’ participation within 
professional development sessions. 

• Although the ACME team has learned to listen to teachers comments rather than dismiss 
them, one challenge is to figure out what merits attention and what does not. 

• Another challenge is to make campus support meaningful, not superficial, such that it will 
support teachers in reflective practice and push their pedagogical skills to levels that 
promote the mathematical understanding of diverse learners. 

• The district needs to make full use of resources on campuses. Although the ACME 
project promotes standards-based curriculum resources and instruction, underutilized 
expertise on campuses is a resource that has the potential to influence improvements in 
mathematics education but is presently falling through the cracks. Ensuring that teacher 
leadership and mentorship in standards-based mathematics is a central part of other 
district initiatives such as Account for Learning is imperative to institutionalize reforms. 

• The ACME team needs to find ways of ensuring that teachers who lack mathematics 
content knowledge learn more. 

• Although teachers seem to be building their capacity to communicate principles of 
standards-based pedagogy, putting this knowledge into practice is one step removed from 
most professional development activities. The addition of peer coaching to the ACME 
project’s repertoire of professional development begins to tackle the need for observation 
and feedback to help teachers improve their pedagogical skills in the classroom. 
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Recommendations 

ACME Professional Development Facilitators 

• Continue to develop team support through collaboration and the drive to improve the 
work of ACME. 

• Continue to enlist the support of campus administrators for standards-based mathematics 
education. 

• Continue to address teachers’ needs through responsiveness, but evaluate priorities. 

Teachers’ Experiences in ACME Professional Development 

• Continue to provide high quality professional development in which teachers have 
opportunities to share with colleagues and experts in standards-based mathematics 
education. 

• Address inactivity among teachers within professional development sessions by asking 
engaging questions that promote participation. 

• Continue to develop strategies of campus support that promote meaningful discussions 
and reflection about standards-based pedagogy that will help teachers improve 
instruction. 

Improving teachers’ knowledge and skills in standards-based mathematics: 

• Focus on helping teachers learn more mathematics content knowledge; determine 
standards with which teachers are uncomfortable, set goals for tackling those concepts in 
professional development, and assess how well those goals were met. 

• Continue to help teachers gain standards-based pedagogical knowledge; set up peer 
coaching networks on campuses to provide teachers observation and feedback on their 
instructional strategies. 

• Continue to actively engage teachers with instructional materials and to have classroom 
teachers model lessons. 

Sustaining Professional Development in Standards-Based Mathematics 

• Establish summer institutes, follow-up during the academic year, and overviews for new 
hires at all grade levels that will continue after the NSF grant ends. 
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Instructions: Please use a #2 pencil to complete this questionnaire. Darken ovals completely, but do not stray into adjacent ovals. 
Be sure to erase completely any stray marks. 



A. Teacher Demographic Information 



1. Are you: 


2a. 


Ethnicity - Are you: 


. O 


Hispanic or Latino 






(Darken one oval.) 


o 


Not Hispanic or Latino 


O Male 










O Female 


2b. 


Race - Are you: 


o 


American Indian or Alaskan Native 






(Choose one or more.) 


o 


Asian 








o 


Black or African-American 








o 


Native Hawaiian or Other Pacific Islander 








o 


White 



3. How many college mathematics courses have you 4. Did your college mathematics coursework include 

completed? (Darken one oval.) the equivalent of at least one semester of: 

(Darken one oval on each line.) 



O 


None 






Yes 


No 


O 


1 semester 


a. 


Number system concepts 


O 


O 


o 


2 semesters 


b. 


Concepts in algebra 


o 


O 


o 


3 semesters 


c. 


Concepts in geometry 


o 


o 


o 


4 semesters 










o 


5 or more semesters 











5. How many years have you taught prior to this school year? (Darken one oval.) 

0-2 3-5 6-10 11-15 16-20 21-25 26 or more 

o o o . , o o o o 



The National Science Foundation's Local Systemic Change (LSC) through Teacher Enhancement 

Program's Core Evaluation 

You have been selected to participate in the nationwide evaluation of the federally-funded Local Systemic Change (LSC) 
program. LSC is a National Science Foundation Teacher Enhancement program that is currently funding more than 60 local 
projects that offer science and mathematics professional development to teachers in 26 states around the country. The cover 
letter accompanying this questionnaire identifies the LSC project in your area, as well as the instructional materials 
that are the focus of that LSC project. 

Each LSC project will administer questionnaires each spring to a randomly-selected sample of teachers who are targeted to 
participate in the local project’s professional development activities. Note that you may be asked to complete this 
questionnaire even if you have not yet participated in the project's professional development; your response is important, 
regardless of whether you have already participated. A small number of randomly-selected teachers in each project is asked to 
provide additional information in interviews, sometimes in conjunction with a classroom visit. In order to continue receiving 
federal funding, each LSC project must participate in this national evaluation. 

Data collection procedures have been developed to ensure high-quality data and protect teacher confidentiality. Your 
responses will be kept strictly confidential; they will be combined with the responses of the other teachers in your project and 
used only for the LSC evaluation. The name label and numbering on this questionnaire are used to help local projects deliver 
questionnaires to the proper teachers and follow up with teachers who have not responded; no information identifying 
individual teachers will be reported under any circumstances. After you complete the questionnaire, you should remove the 
name label and return the questionnaire as specified by your local LSC project. Additional information alx>ut privacy, as well 
as public burden, is provided on page 7 of this questionnaire. 
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B. Teacher Opinions and Preparedness 

6. Please provide your opinion about each of the following statements. 
(Darken one oval on each line.) 



Strongly Disagree 
Disagree 
No Opinion 
Agree 

Strongly Agree 



a. Students generally learn mathematics best in classes with students of similar abilities. 

b. I feel supported by colleagues to try out new ideas in teaching mathematics. 

c. Teachers in this school have a shared vision of effective mathematics instruction. 

d. Teachers in this school regularly share ideas and materials related to mathematics. 

e. Teachers in this school are well-supplied with materials for investigative mathematics 
instruction. 

f. I have time during the regular school week to work with my peers on mathematics 
curriculum and instruction. 

g. I have adequate access to calculators for teaching mathematics. 

h. I have adequate access to computers for teaching mathematics. 

i. I enjoy teaching mathematics. 

j. I am well-informed about the NCTM Standards for the grades I teach. 

k. The mathematics program in this school is strongly supported by local organizations, 
institutions, and/or businesses. 



© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


©. 


© 


© 


© 


© 


© 


© 


© 


© 


© 



7. In the left section, please rate each of the following in terms of its importance for effective mathematics 

instruction in the grades you teach. In the right section, please indicate how prepared you feel to do each one. 
(Darken one oval in each section on each line.) 







Importance 






Preparation 














Not 




Fairly 


Very 




Not 


Somewhat 


Fairly 


Very 


Adequately 


Somewhat 


Well 


Well 


Important 


Important 


Irr^rtant 


Irr^rtant 


Prepared 


Prepared 


Prepared 


Prepare 


a. Provide concrete experience before 


















abstract concepts. 


© 


© 


;© 


© 


© 


© 


© 


© 


b. Develop students' conceptual 
understanding of mathematics. 

c. Take students’ prior understanding 


© 


© . 


© 


© 


o 


© 


© 


© 


into account when planning 
curriculum and insttaiction. 


© 


© 


© 


© 


© 


© 


© 


© 


d. Practice computational skills and 
algorithms. 

e. Make connections between 


© 


© 


© 


©■ 


© 


© 


© 


© 


mathematics and other disciplines, 
f. Have students work in cooperative 


© 


. © 


:© 


; © 


© 


© • 


© r 


© 


learning groups, 
g. Have students participate in 


© 


© 


© 


© 


© 


© 


© 


© 


appropriate hands-on activities, 
h. Engage students in inquiry-oriented 


© 


© 


. © 


© 


© 


© 


© 


© 


activities. 


© 


© 


© 


© 


© 


© 


© 


© 


i. Use calculators. 


© 


© 


© 


© • 


© 


© 


© 


© 


j. Use computers. 

k. Engage students in applications of 


© 


© 


© 


© 


© 


© 


© 


© 


mathem^ics in a variety of contexts. 


© 


© 


© 


© 


© 


© 


© 


© 


1. Use performance-based assessment. 


© 


© 


© 


© 


© 


© 


© 


© 


m. Use portfolios. 

n. Use informal questioning to assess 


© 


© 


© 


© 


© 


© ■ 


© 


© 


student understanding. 


© 





© 


© 







©_ 
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nil 
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np 

np 
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QD 
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m 
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m 

m 

o 
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My principal: (Darken one oval on each line.) 


Strongly 


No 




Strongly 


Disagree Disagree 


Opinion 


Agree 


Agree 


Encourages me to select mathematics content and instructional strategies that 


address individual students' learning. 


© © 


CD 


© 


CD 


Accepts the noise that comes with an active classroom. 

Encourages the implementation of current national standards in mathematics 


© CD 


CD 


© 


CD 


education . 


CD CD 


CD 


© 


CD 


Encourages innovative instructional practices. 

Enhances the mathematics program by providing me with needed materials and 


© CD 


CD 


© 


CD 


equipment. 


CD CD 


CD 


© 


CD 


Provides time for teachers to meet and share ideas with one another. 


CD CD 


CD 


© 


CD 


Encourages me to observe exemplary mathematics teachers. 


CD CD ' 


CD 


© 


CD 


Encourages teachers to make connections across disciplines. 


CD CD 


CD 


© 


CD 


Acts as a buffer between teachers and external pressures (e.g., parents). 


CD CD ‘ 


CD 


© 


CD 



9. Many teachers feel better prepared to teach some subject areas than others. How well prepared do you feel to teach each 
of the following subjects at the grade levels you teach, whether or not they are currently included in your curriculum? 
(Darken one oval on each line.) 





Not 




Fairly 


Very 




Adequately 


Somewhat 


Well 


Well 




Prepared 


Prepared 


Prepared 


Prepared 


a. Science 




CD 


CD 


© 


b. Mathematics 


© 


CD 


CD 


© 


c. Reading/Language Arts 


© 


CD 


CD 


© 


d. Social Studies 




CD 


CD 


© 



10. Within mathematics, many teachers feel better prepared to teach some topics than others. How well prepared do you feel to 
teach each of the following topics at the grade levels you teach, whether or not they are currently included in your 



curriculum? (Darken one oval on each line.) 


Not 




Fairly 


Very 




Adequately 


Somewhat 


Well 


Well 




Prepared 


Prepared 


Prepared 


Prepared 


a. Numeration and number theory 


©■ „ 


CD 


CD 


© 


b. Computation 


© 


CD 


CD 


© 


c. Estimation 


© 


. CD 


CD 


© 


d. Measurement 


© 


CD . 


CD 


© 


e. Pre-algebra 


CD 


CD 


CD 


© 


f. Algebra 


© 


CD 


CD 


© 


g. Patterns and relationships 


© 


CD 


CD 


© 


h. Geometry and spatial sense 


© 


CD 


CD 


© 


i. Data collection and analysis 


© 


CD 


CD 


© 


j. Probability 


© 


CD 


CD 


© 


k. Technology (calculators, computers) in 


support of mathematics © 


CD 


CD 


© 



11. Within the arena of mathematical processes, many teachers feel better prepared to guide and help develop student learning in 
some domains than others. How well prepared do you feel to provide guidance in the following, at the grade levels you teach? 



(Darken one oval on each line.) 


Not 




Fairly 


Very 




Adequately 


Somewhat 


Well 


Well 




Prepared 


Prepared 


Prepared 


Prepared 


a. Problem solving 


© 


CD 


CD 


© 


b. Reasoning and proof 


© 


CD 


CD 


© 


c. Communication (written and oral) 


© 


CD 


CD 


© 


d. Connections within mathematics and from mathematics to other 
disciplines 


. CD 


CD 


CD 


CD 


e. Multiple representations (e.g., concrete models, and numeric, 
graphical, symbolic, and geometric representations) 


© 


CD 


CD 


© 
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12. Please indicate how well prepared you feel to do each of 



the following. (Darken one oval on each line.) 




Not 




Fairly 




Very 




Adequately 


Somewhat 


Well 




Well 






Prepared 


Prepared 


Prepared 


Prepared 


a. Lead a class of students using investigative strategies. 




CD 


CD 


CD 




CD 


b. Manage a class of students engaged in hands-on/project-based 


CD 


CD 


CD 




CD 


work. 




CD 


CD 


CD 




CD 


c. Help students take responsibility for their own learning. 




CD 


CD 


CD 




CD 


d. Recognize and respond to student diversity. 

e. Encourage students' interest in mathematics. 




CD 


CD 


CD 




CD 


f. Use strategies that specifically encourage participation of females 


CD 


CD 


CD 




CD 


and minorities in mathematics. 




CD 


CD 


CD 




CD 


g. Involve parents in the mathematics education of their students. 












Please rate the effect of each of the following on your mathematics instruction. 










(Darken one oval on each line.) 
















Inhibits 






Encourages 


N/A/ 




effective 




Neutral 


effective 


Don't 


instruction 




or mixed 


instruction 


Know 


a. State and/or district curriculum frameworks. 


CD 


CD 


CD 


CD 


CD 


& 


b. State and/or district testing policies and practices. 


CD 


CD 


CD 


CD 


CD 




c. Quality of available instructional materials. 


CD 


CD 


CD 


CD 


CD 


& 


d. Access to calculators for mathematics insthiction. 


CD 


CD 


CD 


CD 


CD 


& 


e. Access to computers for mathematics instruction. 

f. Funds for purchasing equipment and supplies for 


CD 


CD 


CD 


CD 


CD 




mathematics. 


CD 


CD 


CD 


CD 


CD 




g . System of managing instructional resources at the 














district or school level. 


CD 


CD 


"CD 


CD 


CD 


& 


h. Time available for teachers to plan and prepare lessons. 


CD 


CD 


CD 


CD 


CD 


& 


i. Time available for teachers to work with other teachers. 


CD 


CD 


CD 


CD 


CD 


& 


j. Time available for teacher professional development. 


CD 


CD 


CD 


CD 


CD 


& 


kl Importance that the school places on mathematics. 

1. Consistency of mathematics reform efforts with other 


CD . 


!# 






CD- 


& 


school/district reforms. 


CD 


CD 


CD 


CD 


CD 




ni. Public attitudes toward reform. 


CD 


CD 


CD 


CD 




& 


How many of your students' parents do each of the following? 












(Darken one oval on each line.) 






A 


About 




Almost 






None 


Few 


1/2 




All 


a. Volunteer to assist with class activities. 




CD 


CD CD 


CD 


CD 


CD 


b. Donate money or materials for classroom instruction. 




CD 


CD CD 


CD 


CD 


CD 


;c. Attend parent-teacher conferences. 

d. Attend school activities such as PTA meetings and 




CD 


CD- . CD. 


■ CD^^' 


CD .. 




Family Mathematics nights. 

e. Voice support for the use of an investigative approach to 




CD 


CD CD 


CD 


CD 


CD 


mathematics instruction. 




■CD 


CD. CD> 


■ ■ 


CD 


;-cD 


f. Voice support for traditional approaches to mathematics instruction 


/ CD 


CD CD 


CD 


CD 


CD 
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C. Your Mathematics Teaching 

Questions 15-21 ask about your mathematics teaching. Please answer for your first elementary/middle 
school mathematics class of the day. 



15 . What grade level is this class? 
(Darken all ovals that apply.) 



16 . 



[49] 
[45] 

[Ml 

[43] 

[^ 
[^ 
[39] 

H 
[^ 

[35l 

[34] 
[33] 
[32] 

[ID 

[30l 
[29] 
[28] 
[ID 
H 
[D] 
[^ 
[ID 
[ID 
[ID 

DD 
QD 
QD 
QD 
QD 
QD 
QD 
QD 
QD 
QD 
[9] 

m 
m 

E. 

m 

E 

nj 

ERIC 



20 . 



K 

O 



1 

o 



2 

O 



3 

O 



4 

O 



5 

O 



6 

O 



7 

O 



8 

O 



Do you teach in a self-contained classroom (i.e., you are responsible for teaching several subjects to one class)? 
(Darken one oval .) 



O Yes 



O No (Skip to Question 20) 



17 . How many lessons per week do you typically teach mathematics to this class? (Darken one oval.) 

/ Number of Lessons 

0 1 2 3 4 5 

o o ^ ; /o o o o 

18 . Approximately how many minutes is a typical mathematics lesson? (Darken one oval.) 

Average Number of Minutes per Lesson 



19 . 



lOorless 11-20 21-30 31-40 41-50 


51-60 


61-70 


71-80 


81 or more 




0 

p 

:o 

0 

0 


0 


CD 


0 


0 






In how many of the last five school days did you teach each 






Number 


of Days 






of the following in this class? (Darken one oval on each 


None 


One 


Two 


Three 


Four 


Five 


a. Science 


0 


0 , 


0 


0 


0 ■ 


0 


b. Mathematics 


0 


0 


0 


0 


0 


0 


c. Reading/Language Arts 


0 


, Q 


P 


0 


0 


0 


d. Social Studies 


0 


0 


0 


0 


0 


0 



About how often do you do each of the following in your mathematics 




Rarely 


Sometimes 


Often 


All or 


instruction in this class? (Darken one oval on each line.) 




(e.g., a few 
times a 


(e.g., once 
or twice 


(e.g., once 
or twice 


almost all 
mathematics 




Never 


year) 


a month) 


a week) 


lessons 


a. 


Use the LSG-designated instructional materials (see cover 
letter) as the basis of mathematics lesson s. 


CD ■ 


■ ' . 


CD 


Q)'- ■ 


CD 


b. 


Introduce content through formal presentations. 


CD 




CD 


CD 


CD 


c. 


Arran ge seati ng to faci litate student diseussi on . 


CD . 


CD 


CD 


CD \ 


. ■ iCD. 


d. 


Use open-ended questions. 


CD 


CD 


CD 


CD 


CD 


e. 


Require students to explain their reasoning when giving an 
answer. 


CD 


CD 


CD 




;v-'iCD' 


f. 


Encourage students to communicate mathematically. 


0 


CD 


CD 


CD 


CD 


g- 


Encourage students to explore alternative methods for 
solutions. 




CD 


CD 


CD; 


CD 


h. 


Encourage students to use multiple representations (e.g., 
numeric, graphic, geometric, etc.). 


CD 


CD 


CD 


CD 


CD 


i. 


Allow students to work at their own pace. 


CD 


CD 


CD ■ 


CD 


>-qcd; 


j- 


Help students see connections between mathematics and 
other disciplines. 


CD 


CD 


CD 


CD 


CD 


k. 


Use assessment to find out what students know before or 
during a unit. 


CD 


CD 


CD 


CD 




1 . 


Embed assessment in regular class activities. 


CD 


CD 


CD 


CD 


CD 


m. 


Assign mathematics homework. 


CD 


CD 


' ;CD 


^ 0 > V; ' 


• CD 


n. 


Read and comment on the reflections students have written in 
their notebooks or journals. 


CD 


CD 


CD 


CD 


CD 
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About how often do students in this class take part in each of 




Rarely 


Sometimes 


Often 


All or 


the following types of activities as part of their mathematics 




(e.g., a few 


(e.g., once 


(e.g., once 


almost all 


instruction? (Darken one oval on each line.) 




times a 


or twice 


or twice 


mathematic 






Never 


year) 


a month) 


a week) 


lessons 


a. 


Participate in student- led discussions. 


CD 


CD 


CD 


Q 


CD 


b. 


Participate in discussions with the teacher to further 














mathematical understanding. 


CD 


CD 


CD 


(D 


CD 


c. 


Work in cooperative learning groups. 


CD 


CD 


CD 


CD 


CD 


d. 


Make formal presentations to the class. 


CD 


CD 


CD 


CD 


CD 


e. 


Read from a mathematics textbook in class. 


CD 


CD 


CD 


CD 


CD 


f. 


Read other (non-textbook) mathematics-related materials in 


CD 


CD 


CD 


CD 


CD 




class. 


CD 


CD 


CD 


CD 


CD 


g- 


Practice routine computations/algorithms. 
















CD 


CD 


CD 


CD 


CD 


h. 


Review homework/worksheet assignments. 


CD 


CD 


CD 


CD 


CD 


i. 


Use mathematical concepts to interpret and solve word 


CD 


CD 


CD 


CD 


CD 




problems. 


CD 


CD 


CD 


CD 


CD 


j- 


Work on solving a real- world problem. 


CD ‘ 


CD 


CD 


CD 


CD 


k. 


Share ideas or solve problems with each other in small groups. 


CD 


CD 


CD 


CD 


CD 


1. 


Engage in hands-on mathematical activities. 


CD 


CD 


CD 


CD 


CD 


m. 


Play mathematics games. 












n. 


Follow specific instructions in an activity or investigation. 


CD 


CD 


CD 


CD 


CD 






CD 


CD 


CD 


. CD 


CD 


0. 


Design or implement their own investigation. 












P- 


Work on models or simulations. 


CD 


CD 


CD 


CD 


CD 


q- 


Work on extended mathematics investigations or projects (a 


CD 


CD 


CD 


CD 


CD 




week or more in duration). 


CD 


CD 


CD 


CD 


(D 


r. 


Participate in field work. 












s. 


Record, represent and/or analyze data. 


CD 


CD 


CD 


CD 


CD 


t. 


Write a description of a plan, procedure or problem-sol ving 


CD: : 


W 


CD 


^ CD ^ ^ 






process. 












u. 


Write reflections in a notebook or journal. 


CD 


. (D 


CD 


CD 


CD 


V. 


Use calculators or computers for learning or practicing skills. 


CD 


CD 


CD 


CD 


CD 


w. 


Use calculators or computers to develop conceptual 














understanding. 


CD. 


CD 


CD 


CD 


CD 


X. 


Use calculators or computers as a tool (e.g., spreadsheets, data 


CD 


CD 


CD 


CD 


CD 




analysis). 












. y- 


Work on portfolios. 


CD 


^ . .CD 


CD 


CD . 


CD 


z. 


Take short-answer tests (e.g., multiple choice, true/false. 














fill-in -th e-blank) . 


CD 


CD 


CD . 


CD 


CD 


aa. 


Take tests requiring open-ended responses (e.g., descriptions. 


CD 


CD 


© 


CD 


(D 



justifications of solutions). 



D* LSC Professional Development 



Questions 22-27 refer to the NSF-supported Local Systemic Change (LSC) program* Please refer to the cover letter 
accompanying this questionnaire for information about the LSC project activities and designated materials in your district. 
If you have not yet participated in LSC professional development, darken this nvi O and skip to Question 27 . 



22. To what extent is each of the following true of LSC mathematics-related 

professional development in your district? (Darken one oval on each line.) Not 

at all 



To a 

great exter 



a. I aril involved in planning my mathematics-related professional development. 

b. I am encouraged to develop an individual professional development plan to 


CD 


(D ; 


CD 




CD- 


address my needs and interests related to mathematics education, 
c. I am given time to work with other teachers as part of my professional 


CD 


CD 


CD 


CD 


CD 


development. 


<D 


CD 


CD 


CD 


CD 


d. I am given time to reflect on what I've learned and how to apply it to the 


CD 


CD 


CD 


CD 


CD 


classroom. 


CD 


CD 


CD 


CD 


CD 



e. I receive support as I try to implement what I've learned. ^ Q 
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I 

I 

I 

I 

I 

I 

I 



nn 

feTI 



23. 



Approximately how many hours have you spent on formal professional development in mathematics/mathematics 
education as part of the LSC project? (Darken one oval.) 



O 0 
O 1-9 



O 10-19 
O 20-39 



O 40-59 
O 60-79 



O 80-99 
O 100-129 



O 130-159 
O 160-199 



O 200 or greater 



24. 



Please indicate the number of times you have participated in each of the following activities during this school year. 
(Darken one oval on each line.) 

A. Formal Activities 

1. Participated in an LSC academic year study group/discussion group 

2. Was “coached" on my teaching by an LSC lead teacher/staff person based 
on a classroom observation 



B. Informal Activities 



[59] 

H 

[57] 

[56] 

□u 

[54] 

[52] 

[m 

[45] 

[43] 

[3^ 

[M] 

[37] 

[35] 

[3^ 

[3^ 

[m 

I 26 \ 27. Have you been identified as a lead teacher for your distria's NSF-sponsored LSC project? 



1 . 

2 . 

3. 

4. 

5. 



Received assistance from an LSC “lead teacher" in my school 

Received assistance from an LSC staff person in my district 

Received assistance from an LSC-designated mathematician/mathematics 

educator from a college/university/museum/industry 

Read messages in a Listserv discussion sponsored by the LSC 

Posted messages to a Listserv discussion sponsored by the LSC 



25. 



How would you rate the overall quality 
of the LSC professional development? 
(Darken one oval.) 



Very 

Poor 

O 



Poor 

o 



Fair 

O 











7 or 


0 


1-2 


3-4 


5-6 


more 


CD 


CD 


CD 


CD 


CD 


(D 


(D 






_CD^ 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 






Very 






Good 




Good 




Excellent 


0 




0 




0 



26. 



To what extent has participation in LSC mathematics-related professional 
development increased your: (Darken one oval on each line.) 



a. 

b. 

c. 



Mathematics content knowledge 

Understanding of how children think about/leam mathematics 
Ability to implement high-quality mathematics instructional 
materials 



Not 
at all 

CD 

CD 

CD 



To a 

great extent 



CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 
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1998-99 Local Systemic Change 
Teacher Interview 
Middle Years 



1. What grade(s) do you teach? 

2. This district is involved in an NSF-supported local systemic change initiative. * 

To what extent have you participated in those activities (e.g., number of hours/days since 
becoming involved in the project)?^ 

PROBE for both summer and academic year activities. 



3. How do you feel about the professional development provided by the LSC? 

4. How has the LSC affected you and your teaching? 

PROBES: Have there been changes in: 
the content you teach? 
the instructional materials you use? 
the instructional strategies you use? 

your behefs or understanding about effective mathematics/science 
instruction? 

PROBE for examples of changes. 



5. What specific characteristics of the LSC have been most helpful to you? 



6. What aspects have been least helpful? Why? 

7. What else do you need in order to continue improving your mathematics (science) 
instruction? 



You may want to use the local name for the LSC instead of, or in addition to, mentioning NSF, perhaps even giving examples of specific 
activities. 

2 

Only treated teachers who have participated in 20 or more hours of professional development have been included in the random sample for 
teacher interviews. 
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8. Sometimes school and district policies facilitate reform. At other times they get in the way. 
Can you give me some examples of areas in which district pohcies are either supporting the 
LSC reforms or sending the opposite message? [Make note of which areas were mentioned 
by the teacher without probing.] 

PROBES: You’ve mentioned (a couple/a few/several) areas in which you are getting 
(consistent/mixed) messages. I am going to name some areas that other teachers have 
mentioned. If you feel any of these have been either particularly supportive or problematic 
for you, please let me know. [Name only areas not previously mentioned. Be sure to allow 
wait time after each.] 

a. Curriculum frameworks 

b. Instructional materials and supphes 

c. Student grading or testing pohcies 

d. Evaluation of teachers 

e. Time to plan, prepare, or work with other teachers outside of t im e spent at LSC 
sessions 

f. Other teachers’ attitudes toward mathematics (science) education 

g. Principals’ attitudes/expectations 

h. Parents’ attitudes/expectations 



For teachers who have participated in LSC leadership development: 

(If teacher has not participated in LSC leadership development, SKIP to Question 10.) 

9. To what extent have the professional development activities prepared you for your role as a 
teacher leader of mathematics (science) reform in your school or district? 



10. Do you have any other comments you would like to share? 
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1999 Local Systemic Change 
Principal Questionnaire 



Form Approval 
0MB No: 3145-0136 
Expires: August 2001 



Instructions: Please use a #2 pencil to complete this questionnaire. Darken ovals completely, but do not 
stray into adjacent ovals. Be sure to erase completely any stray marks. 

A. Mathematics and Science Instruction 

1. Please provide your opinion about each of the following statements regarding mathematics and science instruction. 
(Darken one oval in each section on each line.) 





Strongly 

Disagree 


Mathematics 

No 

Disagree Opinion Agree 


Strongly 

Agree 


Strongly 

Disagree 


Disagree 


Science 

No 

Opinion 


Agree 


Strong 

Agre 


a. Students generally learn best in classes 
with students of similar abilities. 


d> 


.'CD 


CD 


Ci)':-. ^ 


CD . 


CD ; 


CD 


CD ; ; . 


CD 


CD 


b. I am knowledgeable about current 
national standards in this content area. 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


c. I fed well-prepared to support teachers 
in the implementation of curreri t 
national standards. ^ ^ ^ • 


CD 




CD 


' : 


CD' 


P 


. CD 




CD 


CD 


d. I am willing to accept the noise that 
comes with an active classroom. 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


CD 


e. Encouraging student questions is more 
important than eliciting correct answers 


. CD . 


' CD ' 


CD 


CD: 


.CD 




■■ 


•. CD,:.,;i::: 


•iCD';. 





2. Please provide your opinion about each of the following statements. 

(Darken one oval on each line.) strongly No strongly 

Disagree Disagree Opinion Agree Agree 

ra. Vbcational/technology education should have a Strong mathematics andi^cience 



qompohent. ^ 

b. Students who are not interested in science/mathematics/technology careers should 


CD 




[ p. 






be able to opt out of mathematics and science courses after the 10th or 1 1th grade, 
c. Schools need to provide students who are not interested in 


CD 


CD 


CD 


CD 


CD 


science/mathematics/technology careers course options in mathematics md 
science for all of their high . j : 




. CD:';. 


CD 


■ ■ 


. . ;cD. 


d. Specialized courses in mathematics and science should be available for 












coUeg(^-l:» 


CD 




CD_ 


CD ^ 


CD 



3. 



How would you describe your school’s progress in moving toward excellence in mathematics and science 



education? 




(Darken one oval on each line.) 






Quite far 


Beginning to 


Well along, in 


Approaching 




from ideal 


improve 


improving 


ideal 



a. Mathematics program 


CD 


CD 


m 


CD 




CD 


CD 


b. Science program 




CD 




CD 


d): 


CD 





4. Compared to 5 years ago, which best describes the achievement of students in this school? (Darken one oval on each line.) 





Much 


Somewhat 


About 


Somewhat 


Much 




worse 


worse 


the same 


improved 


improved 


a. Mathematics 


CD; 


;cD, 


CD 


CD: 


CD; 


b. Science 


:Cd; 




CD 


;cD 


CD 
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Please rate each of the follovsdng in terms of its importance for effective mathematics and science instruction. 

(Darken one oval in each section on each line,) 

Mathematics Science 







Not 


Somewhat 


Fairly 


Very 


Not 


Somewhat 


Fairly 






Important Important Important 


Important 


Important 


Important Important 


a. 


Provide concrete experience before abstract 


















concepts. 




© 


© 


© 


© 




© 


b. 


Develop students’ conceptual understanding of 


















the subject. 


© 


© 


© 


© 


© 


© 


© 


c. 


Take students' prior understanding of subject 


















matter into account when planning curriculum 


















and instruction. =: 


© 






..... CD 


:'CD , 


CD 


© 


d. 


Make connections to other disciplines. 


© 


© 


' ©^ 


© 


© 


© 


© 


e. 


Have students work in cooperative learning 














■ 




groups. 


© 


© 


© 


© 


© 


© 


© 


f. 


Have students participate in apprppriate 


- .. . 
















hands-on activities. . ^ ;E: ^ » 




© ' 


'© • 








JS'©;;.: 


g- 


Engage students in inquiry-oriented activities. 


© 


© 


© 


' © 


© 






h. 


Use cdculators; 


CD 


^ © : 


© 


■©. 








i. 


Use computers. 


© 


© 


© 


© 


© 

. . . 


© ' 


© 


j- 


Engage students in applications of subject 












' ■ ’■■ ■ 






matter in a variety of contexts. 


© 


© 


© 


© 


© 


© 


© 


m 


Use performancer^based assessment.^w^^^^ 




CD • 


"CD 


'CD "■■■ 


• CD • 


©^•■ 


©■\: 


1. 


Use portfolios. 


© 


© 


© 


© 


■ ■■■ CD 


© 


© 


m 


Use infornTial questioning to assess student 


















understanding; .. 




© 


.© 


; © •' 


©. 




© 



Please rate the effect of each of the following on mathematics 
instruction in your school. (Darken one oval on each line.) inhibits 

effective 

instruction 



Neutral 
or mixed 



Encourages 

effective 

instruction 



a. 


State and/or disffict curriculum frameworks. 


© 


‘ CD' '■ 


' ■ ©;>'B- ■ ■ 


© 




b. 


State and/or district testing policies and practices. 


© 


© 


© 


© 


^ ' ’ CD ^ 


■Ill 


District/school grading policies and practices. 


•. ©. 




..... ^ ^ 


;cd" 


©B=: 


d. 


District/school structures for recognizing and 
rewarding teachers. 


© 


© 


© 


© 


© 


;^V 


Counseling di^artment polici(^^ a^ 


© 


" "vB;-' © 


CD, ■ ^ 


© ; 




f. 


College placenient tests. 


■ :w 




© ^ 


© 


.... .. j 


g- 


Quality of available instructional materials. 


© 


© 


© 


© 


CD 


h. 


Access to calculators for mathematics instruction; ' 


© 


B \ ^ 


CD • ■ 






i. 


Access to computers for mathematics instruction. 


©' 


© 


© 


© 


"©’ 


j- 


Funds for purchasing equipment: and supplies for 
mathematics. .. ,1^ 


© 




© 


© 


■■-yip 


k. 


System of managing instructional resources at the 















district or school level. . . . 


^'© 


© . 


CD 


© 




1 . 


Time available for teachers to plan and prepare lessons. 


' © 


© 


© 


© 


© 


m. 


Time availifcle for teachers to work wtH other teaches. 


. © 










n. 


Time available for teacher professional development. 


© 


©. 


CD 


© 


©'^ 




Importappejth^^^ . 


• ^'©': 






B©.'„ 


.„B. 


P- 


Consistency of mathematics reform efforts with other 
school/district reforms. 


© 


© 


© 


© 


© 


q- 


^blic attitudes toward ref^^ ' 


"^;©1 









©B' 



Very 



CD 






Q) 



© 

© I 

CD 



© 

CD 

CD 

© 



N/A 

Don't] 

Know 



I 

I 

I 

f 

I 

I 

I 

I 

I 

I 

I 
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7. 



Please rate the effect of each of the following on science 



instruction in your school. 


Inhibits 








Encourages 


N/A 


(Darken one oval on each line.) 


effective 




Neutral 




effective 


Don't 




instruction 




or mixed 




instruction 


Know 


a. State and/or district curriculum frameworks. 


CD ‘ ' 


© 


© 




. © ' ' 




b. State and/or district testing policies and practices. 


CD 


© 


© 


© 


© 




/a Disfrict/schbolg^^ practices. 


CD ' 


© : 


©* " 


© 


'■ ^© 


■ 


d. District/school structures for recognizing and rewarding 














teachers. 


© 


© 


© 


© 


. © 




e. Counseling department polici^ and practices. 


© 


© 


© 


, , © 


CD 




f. College placement tests. 


© . . 


© 


© 


© 


© 




g. Quality of available instructional materials. 


© 


© 


© 


© 


© 


& 


h. Access to calcula,tors for science instruction. 


© • 




© 


CD 


. ' .CD^ 




i. Access to computers for science instruction. 


© 


© 


© 


CD 


© 




j. Funds for purchasing equipment and supplies for science. 


CD 


© 


© 








k. System of managing instructional resources at the district or 












school level. 


CD. . 


' ® 


© 


. © 


© 




1. Time available for teachers to plan and prepare lessons. 


© 


© 


© 


© 


© 




m. Tinie available for teachers to work with other teaches. 


© 


'©\ 


' © ■ ■ 




© 




n. Time available for teacher professional development. 


© 


© 


© 


© 


© 




0 . toportance that the school places on science. 




® 


© 


© 


; ^ '© . 




p. Consistency of science reform efforts with other 














school/district reforms. 


© 


© 


© 


© 


© 




q. Public attitudes toward reform. 


© 


, © 


' ^ 


, © 




& 



Questions 8-9 refer to the NSF-supported Local Systemic Change (LSC) program. Please refer to the cover letter 
accompanying this questionnaire for information about the LSC project activities and designated materials in your district. 



To what extent: 

(Darken one oval on each line.) 



a. 

b. 

c. 

d. 



Are you familiar, with the LSC project in your district? 

Have you been involved in LSC project activities? 

Have parents voiced support for the LSC approach in the classroom? 
Have parents voiced opposition to the LSC approach in the classroom 













N/A 


Not 








To a 


Don't 


at all 








great extent 


Know 


© 


©' 


' © 


© 


’ ' © 


© 


© 


© 


© ' 


© ‘ 


© ' 


© 




^ © 


;© 


© 


© 




© 


© 


■^© 


© 


© 


© 



9. Considering only teachers responsible for teaching the subjects) targeted by the LSC, approximately what percent of 
the teachers in your school: (Darken one oval on each line.) 



0% 


10% 


20% 


30% 


40% 


50% 60% 


70% 


80% 


90% 100% 


;a.'3|f Ha^ in! LSG-professi6hy^ 












|\|| 




••Vy; 


^'/3|:;i:dbvdbpmentfacti\^ 


O 


voV.. 




o : 




MS- 


j®:- 


MSMl 


b. Are implementing at least some of the 


















LSC-designated instructional materials? O 


O 









O 







CD O 



B. Principal Information 

10. Including this year, how many years have you been: 



(Darken one oval on each line.) 


1 


2 


3 


4 


5 


640 


1145 


16-20 


21-25 


More 
than 25 


a. \ A principal?:, J. ; ; . 




V P" ■ 




•• p"; ; 


•• P. 


■ P"'^: 


6 / 


^'\PV" 






b. The principal at this school? 




o ' 


'o 


'(5 


o' 


o' 


o 


o' 


o 


o' 


c. A principal in.t^^^ district? 




:\p V" 


^ CD. V: ‘ 


■'{p 


.■pv- 




•;OVV 
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Horizon Research, Inc. 



'49 



BEST COPY AVAILABLE 



[ 63 ] 

[6^1 

rin 

feol 

[59] 



[ 5 ^ 

[55] 

[13 

[53] 

[13 

[ID 

[sol 

RD 

H 

RD 

R3 

RD 

R3 

RD 

RU 

RD 

R3 

[39] 

R3 

RD 

R3 

RD 

RD 

RD 

RD 

RD 

RD 

RD 

RD 

RD 



RD 

RD 

RD 

RD 

RD 



C. School Characteristics 



~fp!lace label here R;: 



1 






1 1 . How many students 
attend your school? 
(Please enter your 
response as a four- 
digit number and 
then darken the 
appropriate oval in 
each column. For 
example, enter 850 
students as 0850.) 



0O0CD 

0:CDW(I> 

0<I>WQ> 

(DCDCDCD 
Ci) CI) CI) <I) 



students 



©CD®: CD 
© CD CD CD 
CD^CD® CD 


K 


1 


2 
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3 


4 

iilil 
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0 ■ 0, 




7 


8 


9 


10 


11 


12 




0 


0 


0 


0 


p. : 


0 



I 
I 
I 
I 
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12. In what type of community is this school located? (Darken one oval 

O Rural 

'■& Town or Small City 
•'O Suburban 
■0- Urban 

13. This school includes the following grades: (Darken all that apply.) 'I 



14. Approximately what percentage of the students attending this school are: (Please enter each as a three-digit 
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1998-99 Local Systemic Change 
Pre-Classroom Observation Interview 



After you have expressed appreciation to the teacher for allowing you to observe the class, ask 
the following question: 

1. What has this class been doing in mathematics/science recently? 

PROBES: What unit are you working on? 

What instructional materials are you using ? 

2. What do you anticipate doing in your mathematics/science class on the day I will be 
observing? 

PROBE: What do you hope students will learn as a result of the work you have 

planned? 

3. What is the next step for this class? 

4. Is there anything in particular that I should kn ow about the group of students that I will be 
observing? 



* 

Note that the evaluator will need to be thoroughly conversant with the instructional materials designated for use by the LSC in order to complete the 
observation ratings. 
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Tliis form is included for information purposes only. Evaluators \vill need to 
complete the form on the Web. 



1998-99 Local Systemic Change 
Classroom Observation Protocol^ 



Background Information 



Project 



LSC ID" 



Random Sample 
Other, specify 



Backup Sample 



Date of Observation 
Time of Observation: 
Start 



End 



Subject Observed^ 
Grade Level 



Observer 



Observer’s Role in Project: 

Project Evaluator Local Observer 

Lead 

Other 

Section One: Contextual Background and Activities 

In this section, please fill in the circles that best describe the class. For each item, be sure to fill in 
all responses that apply. 



I. Classroom Demographics 

A. What is the total number of students in 
the class at the time of the observation? 
O 15 or fewer 
O 16-20 
O 21-25 
O 26-30 
O 31 or more 



C. Indicate the teacher^ s: 

1 . Gender 

O Male O Female 

2. Race/Ethnicity 

O African-American (not Hispanic origin) 
O American Indian or Alaskan Native 
O Asian or Pacific Islander 
O Hispanic 

O White (not Hispanic origin) 

O Other 



B. What is the approximate percentage of 
white (not Hispanic origin) students in 
this class? 

O 0-10 percent 
O 11-25 percent 
O 26-50 percent 
O 51-75 percent 
O 76-100 percent 

D. If applicable, indicate the teacher aiders: 

1. Gender 

O Male O Female 

2. Race/Ethnicity 

O African-American (not Hispanic origin) 
O American Indian or Alaskan Native 
O Asian or Pacific Islander 
O Hispanic 

O White (not Hispanic origin) 

O Other 



ERIC 



Be sure you have read the “1998-99 Local Systemic Change Classroom Observations: Guidelines for Evaluators” and have completed the “Pre- 
Classroom Observations Interview” before observing the class. 

^ Use the LSC ID number as indicated in the Classroom Observation Sample provided by HRI. 

^ In mathematics/science projects observe the subject for which the teacher was sampled. 
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n. Classroom Context 



A. Rate the adequacy of the physical environment 



1 . Classroom resources: 



0 O O 

1 2 3 

Sparsely equipped 

2. Classroom Space: 

0 O O 

1 2 3 

Crowded 

3. Room arrangement: 

0 O O 

1 2 3 

Inhibited interactions 

among students 



o o 

4 5 

Rich in resources 



o o 

4 5 

Adequate space 



o o 

4 5 

Facilitated interactions 
among students 



B. In a few sentences, describe the lesson you observed. Include where this lesson fits in the overall unit 
of study. 



m. Purposes of Lesson 

A. Indicate the majo/ content area(s) of this lesson or activity. 



O 1 . Numeration and number theory 
O 2. Computation 
O 3. Estimation 
O 4. Measurement 
O 5. Patterns and relationships 
O 6. Pre-algebra 
O 7. Algebra 

O 8. Geometry and spatial sense 
O 9. Functions (including trigonometric 

functions) and pre-calculus concept 
O 10. Data collection and analysis 
Oil. Probability 

0 12. Statistics (e.g., hypothesis tests, 
curve-fitting, and regression) 

0 13. Topics from discrete mathematics 

(e.g., combinatorics, graph theory, 
recursion) 

O 14. Mathematical structures (e.g., vector spaces, 
groups, rings, fields) 

O 15. Calculus 



O 16. Life Science 

(please specify: ) 

O 17. Physical science 

(please specify: ) 

O 18. Earth/space sciences 
O a. Astronomy 
O b. Oceanography 
O c. Geology 
O d. Meteorology 
O e. Environmental sciences 
O 19. Engineering and design principles 
O 20. History of mathematics/science 

0 21. None of the above (please explain) 



“Major” means was used or addressed for a substantial portion of the lesson; if you were describing the lesson to someone, this feature would help 
characterize it. 



o 
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B. Indicate the primary intended purpose(s) of this lesson or activity based on the pre- and/or post- 
observation interviews with the teacher. 

O 1 . Identifying prior student knowledge 

O 2. Introducing new concepts 

O 3. Developing conceptual understanding 

O 4. Reviewing mathematics/science concepts 

O 5. Developing problem-solving skills 

O 6. Learning mathematics/science processes, algorithms, or procedures 

O 7. Learning vocabulary/specific facts 

O 8. Practicing computation for mastery 

O 9. Developing appreciation for core ideas in mathematics/science 

O 10. Developing students’ awareness of contributions of scientists/mathematicians of diverse 
backgrounds 

O 11. Assessing student understanding 

rV. Instructional Materials 

A. Is this lesson based on instructional materials designated for use by this LSC? 

O Yes O No, SKIP to Part V below 



B. Indicate the single set of LSC-designated instructional materials intended to form the basis of this 
lesson (e.g., FOSS; Insights; STC; Investigations in Number, Data, and Space; Connected Math; 
IMP; SEPUP), based on the information provided in the pre-observation interview. 

Please specify. ^ 

C. How closely did the lesson adhere to the instructions provided in the teacher’s manual? 

O Exactly, SKIP to Part V below 

O Almost totally O Mostly O Somewhat O A little O Hardly at all 

D. How did the adaptations affect the quality of the lesson? 

O Helped a lot O Helped a little O Neutral O Hurt a little O Hurt a lot 



V. Classroom Instruction 

A. Indicate the majoi^ way(s) in which student activities were structured. 

O As a whole group O As small groups O As pairs O As individuals 

B. Indicate the majoi^ way(s) in which students engaged in class activities. 

O Entire class was engaged in the same activities at the same time. 

O Groups of students were engaged in different activities at the same time (e.g., centers). 



“Major*’ means was used or addressed for a substantial portion of the lesson; if you were describing the lesson to someone, this feature would help 
characterize it. 
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C. Indicate the majo/ activities of students in this lesson. When choosing an ''umbrella’’ category, be sure 
to indicate subcategories that apply as well. (For example, if you mark "listened to a presentation,” 
indicate by whom.) 

O 1. Listened to a presentation: 

O a. By teacher (would include: demonstrations, lectures, media presentations, extensive procedural instructions) 

O b. By student (would include informzil, as well as formal, presentations of their work) 

O c. By guest speaker/"expert” serving as a resource 

O 2. Engaged in discussion/seminar: 

O a. Whole group 
O b. Small groups/pairs 

O 3. Engaged in problem solving/investigation: 

O a. Worked with manipulatives 
O b. Played a game to build or review knowledge/skills 
O c. Followed specific instructions in an investigation 
O d. Had some latitude in designing an investigation 

O e. Recorded, represented and/or analyzed data 
O f. Recognized patterns, cycles or trends 
O g. Evaluated the validity of arguments or claims 
O h. Provided an informal justification or formal proof 

O 4. Engaged in reading/reflection/written communication about mathematics or science: 

O a. Read about mathematics/science 
O b. Answered textbook/worksheet questions 

O c. Reflected on readings, activities, or problems individually or in groups 
O d. Prepared a written report 

O e. Wrote a description of a plan, procedure, or problem-solving process 
O f. Wrote reflections in a notebook or journal 

O 5. Used technology/audio- visual resource: 

O a. To develop conceptual understanding 
O b. To learn or practice a skill 
O c. To collect data (e.g., probeware) 

O d. As an analytic tool (e.g., spreadsheets or data analysis) 

O e. As a presentation tool 

O f. For word processing or as a communications tool (e.g., e-mail, Internet, Web) 

O 6. Other activities 

O a. Arts and crafts activity 
O b. Listened to a story 
O c. Wrote a poem or story 

O d. Other (Please specify.) 



“Major” means was used or addressed for a substantial portion of the lesson; if you were describing the lesson to someone, this feature would help 
characterize it. 
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D. Comments 

Please provide any additional information you consider necessary to capture the activities or context of 
this lesson. Include comments on any feature of the class that is so salient that you need to get it “on the 
table” right away to help explain your ratings; for example, the class was interrupted by a fire drill, the 
kids were excited about an upcoming school event, or the teacher’s tone was so warm (or so hostile) that it 
was an overwhelmingly important feature of the lesson. 



Section Two: Ratings 

In Section One of this form, you documented what occurred in the lesson. In this section, you are 
asked to rate each of a number of key indicators in four different categories, from 1 (not at aU) to 5 
(to a great extent). You may list any additional indicators you consider important in capturing the 
essence of this lesson and rate these as well. Use your “Ratings of Key Indicators” (Part A) to inform 
your “Synthesis Ratings” (Part B). It is important to indicate in “Supporting Evidence for Synthesis 
Ratings” (Part C) what factors were most influential in determining your synthesis ratings and to give 
specific examples or quotes to illustrate those factors. 

Note that any one lesson is not likely to provide evidence for every single indicator; use 6, “Don't 
know” when there is not enough evidence for you to make a judgment. Use 7, “N/A” (Not 
Applicable) when you consider the indicator inappropriate given the purpose and context of the 
lesson. Section Two concludes with ratings of the likely impact of instruction, and a capsule 
description of the lesson. 
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I. Design 



ERIC 



Not 

at 

all 



To a 
great 
extent 



A. Ratings of Key Indicators 

1. The design of the lesson incorporated tasks, roles, and 
interactions consistent with investigative mathematics/science. 

2. The design of the lesson reflected careful planning and 
organization. 

3. The instructional strategies and activities used in this 
lesson reflected attention to students’ experience, 
preparedness, and/or learning styles. 

4. The resources available in this lesson contributed to 
accomplishing the purposes of the instruction. 

5. The instructional strategies and activities reflected attention 
to issues of access, equity, and diversity for students 

(e.g., use of “wait time,” cooperative learning, language- 
appropriate strategies/materials). 

6. The design of the lesson encouraged a collaborative 
approach to learning. 

7. Adequate time and structure were provided for reflection. 

8. Adequate time and structure were provided for wrap-up 
and closure. 

9. Formal assessments of students were consistent with 
investigative mathematics/science. 

10. Design for future instruction takes into account what 
transpired in the lesson. 



11 . 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 



Don’t 

know 



6 

6 



7 

7- 



B. Synthesis Rating 



C. Supporting Evidence for Synthesis Rating 
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1 


2 


3 


4 


■ 


Design of the lesson not at 
all reflective of best 
practice in 
mathematics/science 
education 








Design of the lesson 
extremely reflective of 
best practice in H 

mathematics/science H 

education 
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n. Implementation 



A. Ratings of Key Indicators 

1. The instruction was consistent with the underlying 
approach of the instructional materials designated 
for use by the LSC. 

2. The instructional strategies were consistent with 
investigative mathematics/science. 

3. The teacher appeared confident in his/her ability to teach 
mathematics/science. 



Not 

at 

all 



To a 

great Don’t 
extent know N/A 



1 2 3 4 5 6 7 

1 2 3 4 5 6 7 



1 2 3 4 5 



6 7 



4. The teacher’s classroom management style/strategies 

enhanced the quality of the lesson. 1 2 3 4 5 

5. The pace of the lesson was appropriate for the developmental 

levels/needs of the students and the purposes of the lesson. 1 2 3 4 5 



6 7 

6 7 



6. The teacher took into account prior knowledge of students. 



1 2 3 4 5 



6 7 



7. The teacher’s questioning strategies were likely to enhance the 
development of student conceptual understanding/problem solving 
(e.g., emphasized higher order questions, appropriately used 
“wait time,” identified prior conceptions and misconceptions). 



1 2 3 4 5 



6 7 



8. The lesson was modified as needed based on teacher 
questioning or other student assessments. 



9. 



1 2 3 4 5 

1 2 3 4 5 



6 7 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Implementation of the 
lesson not at all reflective 
of best practice in 
mathematics/science 
education 








Implementation of the 
lesson extremely reflective 
of best practice in 
mathematics/science 
education 



C. Supporting Evidence for Synthesis Rating 



BEST COPY AVAILABLE 
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m. Mathematics/Science Content 



A. Ratings of Key Indicators 



Not 

at 

all 



To a 

great Don’t 
extent know 



1. The mathematics/science content was significant and worthwhile. 



2 3 4 5 



6 7 



2. The mathematics/science content was appropriate for the 
developmental level of the students in this class. 



2 3 4 5 



6 7 



3. Students were intellectually engaged with important ideas 
relevant to the focus of the lesson. 



2 3 4 5 



6 7 



4. Teacher-presented information was accurate. 

5. The teacher displayed an understanding of niathematics/science 
concepts (e.g., in his/her dialogue with students). 

6. Mathematics/science was portrayed as a dynamic body of 
knowledge continually enriched by conjecture, investigation 
analysis, and/or proof(justification. 

7. Elements of mathematical/science abstraction (e.g., symbolic 
representations, theory building) were included when it was 
important to do so. 



1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 



8. Appropriate connections were made to other areas of mathematics/ 

science, to other disciplines, and/or to real-world contexts. 1 2 3 4 5 

9. The degree of closure or resolution of conceptual understanding 
was appropriate for the developmental levels/needs of the 

students and the purposes of the lesson. 1 2 3 4 5 



6 7 

6 7 

6 7 



6 7 

6 7 



6 7 



10 . 



2 3 4 5 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Mathematics/science 
content of lesson not at all 
reflective of current 
standards for 
mathematics/science 
education 








Mathematics/science 
content of lesson 
extremely reflective of 
current standards for 
mathematics/science 
education 



C. Supporting Evidence for Synthesis Rating 



BEST COPY AVAILABLE 
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rV, Classroom Culture 



Al. Ratings of Key Indicators 



Not 

at 

all 



To a 

great Don’t 
extent know N/A 



1. Active participation of all was encouraged and valued. 



2 3 4 5 



6 7 



2. There was a climate of respect for students’ ideas, 
questions, and contributions. 

3. Interactions reflected collaborative working relationships 
among students (e.g., students worked together, talked with 
each other about the lesson). 



1 2 3 4 5 



6 7 



1 2 3 4 5 



6 7 



4. Interactions reflected collaborative working relationships 
between teacher and students. 



2 3 4 5 



6 7 



5. The climate of the lesson encouraged students to generate 
ideas, questions, conjectures, and/or propositions. 



1 2 3 4 5 



6 7 



6. Intellectual rigor, constructive criticism, and the challenging 
of ideas were valued. 



2 3 4 5 



6 7 



7. 



2 3 4 5 



A2. Respect for Diversity 

Based on the culture of a classroom, observers are generally able to make inferences about the extent to 
which there is an appreciation of diversity among students (e.g., their gender, race/ethnicity, and/or cultural 
background). While direct evidence that reflects particular sensitivity or insensitivity toward diversity is not 
often observed, we would like you to document any examples you do see. If any examples were observed, 
please check here □ and describe below: 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Classroom culture 
interferes with student 
learning 








Classroom culture 
facilitates the learning of 
all students 



C. Supporting Evidence for Synthesis Rating 
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V. Overall Ratings of the Lesson 



A. Likely Impact of Instruction on Students’ Understanding of Mathematics/Science 

While the impact of a single lesson may well be limited in scope, it is important to judge whether the lesson 
is helping move students in the desired direction. For this series of ratings, consider all available 
information (i.e., your previous ratings of design, implementation, content, and culture, and the pre- and 
post-observation interviews with the teacher) as you assess the likely impact of this lesson. Feel free to 
elaborate on ratings with comments in the space provided. 

Select the response that best describes your overall assessment of the likely effect of this lesson in each of the 
following areas. 



1 . Students’ understanding of mathematics/science as a dynamic 
body of knowledge generated and enriched by investigation. 

2. Students’ understanding of important mathematics/science 
concepts. 

3. Students’ capacity to carry out their own inquiries. 

4. Students’ ability to apply or generalize skills and concepts to 
other areas of mathematics/science, other disciplines, and/or 
real-life situations. 

5. Students’ self-confidence in doing mathematics/science. 

6. Students’ interest in and/or appreciation for the discipline. 



Mixed or 
Negative Neutral 



Positive 



effect 




effect 




effect 


know 


N/A 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


d 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 



Don’t 



Comments (optional): 
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B. Capsule Description of the Quality of the Lesson 

In this final rating of the lesson, consider all available information about the lesson, its context and 
purpose, and your own judgment of the relative importance of the ratings you have made. Select the 
capsule description that best characterizes the lesson you observed. Keep in mind that this rating is not 
intended to be an average of all the previous ratings, but should encapsulate your overall assessment of 
the quality and likely impact of the lesson. Please provide a brief rationale for your final capsule 
description of the lesson in the space provided. 

O Level 1: Ineffective Instruction 

There is little or no evidence of student thinking or engagement with important ideas of 
mathematics/science. Instruction is unlikely to enhance students’ understanding of the discipline or 
to develop their capacity to successfully “do” mathematics/science. Lesson was characterized by 
either (select one below): 

O Passive ‘‘Learning” 

Instruction is pedantic and uninspiring. Students are passive recipients of information from the 
teacher or textbook; material is presented in a way that is inaccessible to many of the students. 

O Activity for Activity’s Sake 

Students are involved in hands-on activities or other individual or group work, but it appears to 
be activity for activity’s sake. Lesson lacks a clear sense of purpose and/or a clear link to 
conceptual development. 

O Level 2: Elements of Effective Instruction 

Instruction contains some elements of effective practice, but there are substantial problems in the 
design, implementation, content, and/or appropriateness for many students in the class. For example, 
the content may lack importance and/or appropriateness; instruction may not successfully address the 
difficulties that many students are experiencing, etc. Overall, the lesson is quite limited in its 
likelihood to enhance students’ understanding of the discipline or to develop their capacity to 
successfully “do” mathematics/science. 

O Level 3: Beginning Stages of Effective Instruction (Select one below.) 

O Low 3 O Solid 3 O High 3 

Instruction is purposeful and characterized by quite a few elements of effective practice. Students 
are, at times, engaged in meaningful work, but there are some weaknesses in the design, 
implementation, or content of instruction. For example, the teacher may short-circuit a planned 
exploration by telling students what they “should have found”; instruction may not adequately 
address the needs of a number of students; or the classroom culture may limit the accessibility or 
effectiveness of the lesson. Overall, the lesson is somewhat limited in its likelihood to enhance 
students’ understanding of the discipline or to develop their capacity to successfully “do” 
mathematics/ science . 

O Level 4: Accomplished, Effective Instruction 

Instruction is purposeful and engaging for most students. Students actively participate in 
meaningful work (e.g., investigations, teacher presentations, discussions with each other or the 
teacher, reading). The lesson is well-designed and the teacher implements it well, but adaptation of 
content or pedagogy in response to student needs and interests is limited. Instruction is quite likely to 
enhance most students' understanding of the discipline and to develop their capacity to successfully 
“do” mathematics/science. 

O Level 5: Exemplary Instruction 

Instruction is purposeful and all students are highly engaged most or all of the time in meaningful 
work (e.g., investigation, teacher presentations, discussions with each other or the teacher, reading). 
The lesson is well-designed and artfully implemented, with flexibility and responsiveness to students’ 
needs and interests. Instruction is highly likely to enhance most students' understanding of the 
discipline and to develop their capacity to successfully “do” mathematics/science. 
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1998-99 Local Systemic Change 
Post-Classroom Observation Interview 



After you have expressed appreciation to the teacher for allowing you to observe the class, ask 
the following questions: 

1. Were there any ways in which the lesson was different from what you had plaimed? 

2. What did this lesson teU you about what your students are learning and still need to learn 
in mathematics/science? 

PROBE; How do you plan to further assess the students’ learning? 

3. What challenges have you faced in encouraging your students to be actively engaged in 
this mathematics/science class? 

PROBE: How have you approached these challenges? 

4. What is the next step for this class? 
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1998-99 Local Systemic Change 
Pre-Observation Interview with 
Professional Development Facilitator 



1. Please talk with me briefly about the primary purposes of the professional development 
session I will be observing. 

PROBE: What do you hope participants will gain as a result of their participation 

in this session?' 



2. What do you anticipate happening during the session I will be observing? 

PROBES: Will the session include any of the materials the LSC has designated for classroom 
use? 

If so, how will they be used? 



3. How does this session fit into the sequence of professional development experiences planned 
for this district’s teachers? 

PROBES: What experiences have these participants had with the LSC prior to this 
session? 

What will they do next? 



4. Tell me a little about your background as it relates to the session you will be facilitating. 



5. Is there anything in particular that I should know about the participants who will be attending 
this session? 



* Several of the ratings on the Professional Development Observation Protocol require an understanding of the 
intended purposes of the session. If the facilitator is not explicit in describing the purposes of the session, further 
probes may be needed. Additional probes might include direct questions about the extent to which the session is 
intended to enhance participants’ content knowledge, to explore pedagogical strategies/instructional materials or to 
explore strategies/issues/roles for teacher leaders, principals, or others in leadership positions. Refer to Section 
One, niA on the Professional Development Obseivation Protocol for a list of potential purposes. 
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NOTE: 



This form is included for information purposes only. Evaluators will need to 
complete the form on the Web. 



1998-99 Local Systemic Change 
Professional Development Observation Protocol^ 

Background Information 



Project 



Location 



Date of Observation 

If you are submitting two professional development observations 
for this date, indicate whether this was the first or second 
session observed, O O 2“^ 



Observer 



Observer's Role in Project: 



O Lead Evaluator 



Approximate Duration of Observation^: 

O 1 hour O 3 hours 

O 2 hours O half day 

O Other 



Subject Targeted by session O Mathematics O Science O Both Mathematics and Science 



Section One: Contextual Background And Activities 

In this section, please fill in the circles that best describe the session. For each item, be sure to fill in all 
responses that apply. 

I. Session Demographics 

A« What is the total number of participants attending this session? 

O 1-5 O 6-10 O 11-20 O 21-50 O 51-100 O More than 100 

Please describe the targeted subject(s)/grade level(s)/audience for this professional development session. 

1. This session was intended to improve the teaching of: (select all that apply) 

O Elementary science O Elementary mathematics 

O Middle grades science O Middle grades mathematics 
O High school science O High school mathematics 

2. Participants were: 

O Lead teachers for the LSC projects 
O Other (non-lead) teachers 
O Administrators 

O Other (Please specify.) 



BESfT COPY AVAILABLE 



Be sure you have read the “ 1 998-99 Local Systemic Change Professional Development Observations: Guidelines for Evaluators” and have completed the 
“Pre-Observation Interview with Professional Development Facilitator” before observing the session. 

2 

The observation recorded on this form should be no less than one hour and no more than half a day. 
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C* Please describe the major presenters/facilitators^ for this particular one-hour to half-day 
professional development session. 



1 . 



Indicate the number of presenters/facilitators in each gender and race/ethnicity category. 





African-American 
(not Hispanic-origin) 


American Indian 
or Alaskan Native 


Asian or 
Pacific Islander 


Hispanic 


White (not 
Hispanic origin) 


Other 


M^e 














Female 















2. Indicate the number of 



jresenters/facilitators for this particular session with each affiliation. 



Regular 

Full-Time 

or 

Part-Time 

Classroom 

Teachers 


Teachers 

on 

Special 

Assignment'* 


District 

Mathematics/ 

Science 

Supervisor 


Other 

District 

Personnel 


University 

Mathematics/ 

Science 

Faculty 


University 

Mathematics/ 

Science 

Education 

Faculty 


Business 

Industry 

Mathematicians/ 

Scientists 


Other 

Non- 

District 

Personnel 



















n. Session Context 

In a few sentences, describe the session you observed. Include: (a) whether the observation covered a partial or 
complete session, (b) whether there were multiple break-out sessions, and (c) where this session fits in the project's 
sequence of professional development for those in attendance. 

m. Session Focus 

A. Indicate the primary intended purpose(s) of this professional development session based on the information 
provided by the project staff or session organizer/facilitator, 

O 1. Increasing mathematics/science content knowledge of participants. {Be sure to complete Category III: 
Mathematics/Science Content and Category VILA: Likely Impact on Participants ' Capacity to Provide 
High-Quality Mathematics/Science Education, in Section Two of the protocol.) 

O 2. attention to classroom* pedagogy/instructional materials. {Be sure to complete Category IV: 

Exploring Pedagogy/Instructional Materials and Category VILA: Likely Impact on Participants' Capacity 
to Provide High-Quality Mathematics/Science Education, in Section Two of the protocol.) 

O a. Creating a vision of effective mathematics/science instruction 
O b. Understanding student thinking/leaming about mathematics/science content 
O c. Learning how to use specific instructional materials in the classroom 
O d. Learning how to use technology in the classroom. 

O e. Learning pedagogical/classroom management strategies 
O f. Considering issues of access, equity, and diversity 
O g. Designing or scoring student assessments 

O h. Considering issues of scope and sequence (e.g., K-12 curricular frameworks) 

O 3. Explicit attention to strategies/issues/roles of teacher leaders, principals, or others in leadership positions. 
{Be sure to complete Category V: Leadership Content and Category VI LB: Likely Impact on Participants' 
Leadership Capacity, in Section Two of the protocol.) 

O 4. Other major purposes: 

O a. Orientation to the project 

O b. Assessing participants’ knowledge/skills 

O c. Building professional networks among educators 

O d. Promoting/exploring reflective practice 

O e. Developing the capacity of participants to use technology 

O f. Involving administrators and/or other school/district personnel in the reform process 



^ In some instances this may not be appropriate, e.g., a session in which a group of teachers meets after school to discuss their action research projects may have no 
presenters or facilitators. In these instances, please leave the presenters/facilitators cells blank. 

^ Defined as teachers released full-time from classroom responsibilities to work on assignments such as the LSC project. 
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B. Indicate the majot^ mathematics/science content area(s) addressed in this professional development session, whether 
increasing content knowledge was a stated purpose or the mathematics/science content was simply a vehicle for 
achieving other purposes. 



o 


1. 


Numeration and number theory 


o 


16. 


Life Science (Please specify. 'I 


o 


2. 


Computation 


o 


17. 


Physical science (Please specify.') 


o 


3. 


Estimation 


o 


18. 


Earth/space sciences 


o 


4. 


Measurement 






O a. Astronomy 


o 


5. 


Patterns and relationships 






O b. Oceanography 


o 


6. 


Pre-algebra 






O c. Geology 


o 


7. 


Algebra 






O d. Meteorology 


o 


8. 


Geometry and spatial sense 






O e. Environmental science 


o 


9. 


Functions (including trigonometric 


o 


19. 


Engineering and design principles 






functions) and pre-calculus 


o 


20. 


History of mathematics/science 






concepts 


o 


21. 


Mathematics/science as a way of knowing 


o 


10. 


Data collection and analysis 






(e.g., inquiry, problem solving) 


o 


11. 


Probability 








o 


12. 


Statistics (e.g., hypothesis tests. 












curve-fitting, and regression) 


□ 


Mathematics/science concepts were not included 


o 


13. 


Topics from discrete mathematics 




as either an explicit focus or a vehicle for achieving other 






(e.g., combinatorics, graph theory. 




professional development purposes 






recursion) 








o 


14. 


Mathematical structures (e.g., 
vector spaces, groups, rings, 
fields) 








o 


15. 


Calculus 









rv. Professional Development Activities 



A. Were any of the instructional materials intended for classroom use as part of the LSC (e.g., FOSS; Insights; STC; 
SEPUP; Investigations in Number, Data, and Space; Connected Math; IMP; Core Plus) a focus of the professional 
development session? 



O No 

O Yes Please specify. 

B. Indicate the majoi^ activities of participants in this session. When choosing an "umbrella" category, be sure to indicate 
subcategories that apply as well. For example, if you mark "formal presentations," indicate by whom. 



O 1 . Listened to a formal presentation by: 

O a. Session presenter/facilitator 
O b. Participant(s) 



O 2. Engaged in discussions/seminars/reporting out structured 
as: 

O a. Entire group led by presenter/facilitator 
O b. Entire group led by participant(s) 

O c. Subsets of the group 



O 3. Engaged in problem solving/investigation focusing on disciplinary content, pedagogy, and/or reform issues 
P 4. Read about disciplinary content, pedagogy, or reform issues 
O 5. Wrote about disciplinary content, pedagogy, or reform issues 



BEST COPY AVAILABLE 



“Major” means was used or addressed for a substantial portion of the session; if you were describing the session to someone, this feature would help characterize it, 
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C. Indicate the major professional development approaches used in this session/ 

O Workshop/institute/course/seminar 
O Receiving formal professional development via technology 
O Study groups/“kit clubs’Vdiscussion groups/school-based meetings 
O Coaching/mentoring 

O Other: 

D. Comments 

Please provide any additional information you consider necessary to capture the activities or context of this professional 
development session. Include comments on any feature of the session that is so salient that you need to get it "on the table” 
right away to help explain your ratings. 



Section Two: Ratings 

In Section One of this form, you documented what occurred in the session. In this section, you are asked to use that 
information, as well as any other pertinent observations, to rate each of a number of key indicators in six different categories, 
from 1 (not at all) to 5 (to a great extent). 

Note that any one session is not likely to provide evidence for every single indicator; use 6, “Don't know” when there is not 
enough evidence for you to make a judgment. Use 7, “N/A” (Not Applicable) when you consider the indicator inappropriate 
given the purpose and context of the session. For example, a session that focuses on engaging teachers in mathematics/science 
inquiry may choose not to address classroom applications. In that case, key indicator #7 for Design, ‘The design of the session 
provided opportunities for teachers to consider classroom applications of resources, strategies, and techniques,” would be rated 
“N/A,” rather than “not at all.” 

Similarly, there may be entire rating categories that are not applicable to a particular session. For example, categories in, TV, 
and V (Content) and Overall Ratings VILA (Impact on Participants’ Capacity to Provide High Quality Mathematics/Science 
Education) and VIIB (Impact on Participants’ Leadership Capacity) each have a box to check when the entire rating category is 
judged to be inappropriate for the session^. Categories I (Design), II (Implementation), and VI (Culture of the Professional 
Development Session) are ones in which specific indicators may be “not applicable,” but the overall category should routinely be 
rated for any observation. 

Note that you may list any additional indicators you consider important in capturing the essence of this session and rate these as 
well. 

Use your “Ratings of Key Indicators” (Part A) to inform your “Synthesis Ratings” (Part B). It is important to indicate in 
“Supporting Evidence for Synthesis Ratings” (Part C) what factors were most influential in determining your synthesis ratings 
and to give specific examples or quotes to illustrate those factors. Section Two concludes with ratings of the likely impact of 
professional development, and a capsule description of the session. 



® Observers should refer to the Annotated Guide to the Professional Development Observation Protocol for descriptions of each of these professional development 
approaches. 

In most cases, the categories you rate will be consistent with the purposes marked in Section One. Part in. A. 1 through 3. 
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L Design 

A. Ratings of Key Indicators 



1 . 

2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 

9. 

10 . 

11 . 

12 . 



The design of the session incorporated tasks, roles, and 
interactions consistent with a spirit of investigation. 

The instructional strategies and activities used in this 
session reflected attention to participants’ experience, 
preparedness, and/or learning styles. 

The session effectively built on participants’ knowledge of 
content, teaching, learning, and/or the reform process. 

The strategies in this session were appropriate for accomplishing 
the purposes of the LSC professional development. 

The design of the session reflected careful planning and 
organization. 

The design of the session included “framing” the activity 
to help participants understand the purpose of the session and 
where it fits into the larger professional development picture. 

The design of the session encouraged a collaborative 
approach to learning. 

The design, of the session provided opportunities for teachers 
to consider classroom applications of resources, strategies, 
and techniques. 

Adequate time and structure were provided for “sense-making,” 
including reflection about concepts, strategies, issues, etc. 

Adequate time and structure were provided for participants 
to share experiences and insights. 



Not 

at 

all 



1 



Adequate time and structure were provided for wrap-up and closure. 1 

1 



To a 
great 
extent 



4 5 



4 5 



4 

4 

4 



Don’t 

know 



N/A 



7 

7 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Design of the session not 
at all reflective of best 
practice for professional 
development. 








Design of the session 
extremely reflective of 
best practice for 
professional development. 



C. Supporting Evidence for Synthesis Rating 



BEST COPYAVAIUBLE 
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n. Implementation 



A. Ratings of Key Indicators 

1 . Formal presentations included in the session were carried 
out effectively. 



Not 

at 

all 



To a 

great Don’t 
extent know N/A 



1 2 3 4 5 



6 7 



2. The facilitator(s)’ contributions during the course of the session 
enhanced the quality of the session. 



2 3 4 5 



6 7 



3. The facilitator(s) effectively modeled questioning strategies that are 
likely to enhance the development of conceptual understanding 
(e.g., emphasis on higher-order questions, appropriate use of 

“wait time,” identifying prior conceptions and misconceptions.) 1 2 3 4 5 

4. The facilitator(s)’ background, experience, and/or expertise 

enhanced the quality of the session. 1 2 3 4 5 



5. The facilitator(s)’ management style enhanced the 

quality of the session. 1 2 3 4 5 

6. The pace of the session was appropriate for the purposes of 

the professional development and the needs of adult learners. 1 2 3 4 5 



7. The session modeled effective assessment strategies. 



2 3 4 5 



8 . 



2 3 4 5 



6 7 

6 7 

6 7 

6 7 

6 7 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Implementation of the 
session not at all reflective 
of best practice for 
professional development. 








Implementation of the 
session extremely 
reflective of best practice 
for professional 
development 



C. Supporting Evidence for Synthesis Rating 



BEST COPY AVAILABLE 
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m. Mathematics/Science Content 



Complete this category if: a) increasing mathematics/science content knowledge was a key purpose of the session; b) 
mathematics/science content was a vehicle for accomplishing other professional development purposes; or c) inadequate 
coverage in this area acted as a barrier to accomplishing other stated purposes of the session. If none of these apply, 
check here □ and skip to category IV. 



A. Ratings of Key Indicators 

1. Mathematics/science content was appropriate for the purposes 
of the professional development session and the backgrounds 
of the participants. 

2. Mathematics/science content was sound and appropriately 
presented/explored. 

3. Participants were intellectually engaged with important 
ideas relevant to the focus of the session. 

4. Facilitator(s) displayed an understanding of mathematics/science 
concepts (e.g., in their dialogue with participants). 

5. Mathematics/science was portrayed as a dynamic body of 
knowledge continually enriched by conjecture, investigation, 
analysis, mdJqr proof(justification. 

6. Depth and breadth of attention to mathematics/science content was 
appropriate for the purposes of the session and participants’ needs. 

7. Elements of mathematical/scientific abstraction (e.g., symbolic 
representations, theory building) were included when it was 
important to do so. 

8. Appropriate connections were made to other areas of mathematics/ 
science, to other disciplines, and/or to real-world contexts. 

9. Degree of closure or resolution of mathematics/science 
conceptual understanding was appropriate for the purposes of the 
session and the needs of adult learners. 



10 . 



Not 

at 

all 



1 2 3 



2 

2 



To a 
great 
extent 



4 

4 



Don’t 

know 



N/A 



B. 



C. 



Synthesis Rating 



1 


2 


. 3 


4 


5 


Mathematics/science 
content of session not at 
all reflective of current 
standards for 
mathematics/science 
education 








Mathematics/science 
content of session 
extremely reflective of 
current standards for 
mathematics/science 
education 



Supporting Evidence for Synthesis Rating 



BEST COPY AVAILABLE 
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IV. 



Exploring Pedagogy/lnstructional Materials 



Complete this category if: a) exploring classroom practice/instructional materials was a key purpose of the session; or b) 
lack o^inadequate coverage in this area acted as a barrier to accomplishing other stated purposes of the session. If 
neither of these apply, check here □ and skip to category V. 



A. Ratings of Key Indicators 

1. Depth and breadth of attention to student thinking/learning were 
appropriate for the purposes of the session and participants’ needs. 



Not 

at 

all 



To a 

great Don’t 
extent know N/A 



1 2 3 4 5 



6 7 



2. Depth and breadth of attention to classroom 

strategies were appropriate for the purposes of the session and 

participants’ needs. 1 2 3 4 5 6 7 

3. Depth and breadth of attention to instructional materials intended for 
classroom use were appropriate for the purposes of the session and 

participants’ needs. 1 2 3 4 5 6 7 



4. Facilitator(s) displayed an understanding of pedagogical 
concepts (e.g., in their dialogue with participants). 

5. Participants were intellectually engaged with important 
ideas relevant to classroom practice. 

6. Degree of closure or resolution of conceptual understanding ‘ 
about classroom practice was appropriate for the purposes 

of the session and the needs of adult learners. 



1 2 3 4 5 



6 7 



1 2 3 4 5 



6 7 



1 2 3 4 5 



6 7 



7. 



2 3 4 5 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Pedagogical content of 
session not at all reflective 
of current standards for 
mathematics/science 
education 








Pedagogical content of 
session extremely 
reflective of current 
standards for 
mathematics/science 
education 



C. Supporting Evidence for Synthesis Rating 









Horizon Research, Inc. 



1998-99 Core Evaluation Manual: Professional Development Observation Protocol - Page 8 



March 1999 



V. Leadership Content 



Complete this category only if exploring strategies/issues/roles of teacher leaders, principals, or others in leadership 
positions was a key purpose of the session. If not, check here □ and skip to category VI. 



A. 


Ratings of Key Indicators 


Not 






To a 










at 






great 


Don’t 




1. 


Information on principles of effective staff development was 


all 






extent 


know 


N/A 




sound and appropriately presented/explored. 


1 


2 


3 


4 


5 


6 


7 


2. 


Information on strategies for mentoring/coaching peers was 


















sound and appropriately presented/explored. 


1 


2 


3 


' 4 


5 


6 


7 


3. 


Information on how to be a reform advocate at school/district 


















level was sound and appropriately presented/explored. 


1 


2 


3 


4 


5 


6 


7 


4. 


Facilitator(s) displayed an understanding of leadership concepts 


















(e.g., in their dialogue with participants). 


1 


2 


3 


4 


5 


6 


7 


5. 


Participants were intellectually engaged with important 


















ideas relevant to the focus of the session. 


1 


2 


3 


4 


5 


6 


7 


6. 


Participants were given adequate and appropriate opportunity to 


















consider how the content of the session applies to their particular 


















leadership roles. 


1 


2 


3 


4 


5 


6 


7 


7. 




1 


2 


3 


4 


5 







B. Synthesis Rating 



1 


2 


3 


4 


5 


Leadership content not at 
all appropriate for 
preparing participants to 
be school/district leaders 
of mathematics/science 
education 








Leadership content highly 
appropriate for preparing 
participants to be 
school/district leaders of 
mathematics/science 
education 



C. Supporting Evidence for Synthesis Rating 






Horizon Research, Inc. 
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VI. Culture of the Professional Development Session 

Al. Ratings of Key Indicators 

1 . Active participation of all was encouraged and valued. 

2. There was a climate of respect for participants’ experiences, 
ideas, and contributions. 

3. Interactions reflected collaborative working relationships 
among participants. 

4. Interactions reflected collaborative working relationships 
between facilitator(s) and participants. 

5. Participants were encouraged to generate ideas, questions, 
conjectures, and propositions. 

6. Intellectual rigor, constructive criticism, and the challenging 
of ideas were valued. 

7. Investigation and risk-taking were valued. 



Not 

at 

all 



To a 
great 
extent 



1 2 3 4 5 

1 2 3 4 5 

1 2 '3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 



Don’t 

know 



6 

6 



N/A 

7 



A2. Respect for Diversity 

Based on the culture of a professional development session, observers are generally able to make inferences about the 
extent to which there is an appreciation of diversity among participants (e.g., their gender, race/ethnicity, and/or 
cultural background). While direct evidence that reflects particular sensitivity or insensitivity toward diversity is not 
often observed, we would like you to document any examples you do see. If any examples were observed, please 
check here □ and describe below: 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Culture of the session 
interferes with 
engagement of 
participants as members of 
a professional learning 
community 








Culture of the session 
facilitates engagement of 
participants as members of 
a professional learning 
community 



C. Supporting Evidence for Synthesis Rating 
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Vn. Overall Ratings of the Session 



While the impact of a single professional development session may well be limited in scope, it is important to judge whether 
the session is helping move participants in the desired direction. For ratings in Sections A and B below, consider all available 
information (i.e., your previous ratings of design, implementation, content, and culture; related interviews; and your 
knowledge of the overall professional development program) as you assess the likely impact of this session. Feel free to 
elaborate on ratings with comments in the space provided. 

A. Likely Impact on Participants’ Capacity to Provide High Quality Mathematics/Science 
Education 

Consider the likely impact of this session on the participants’ capacity to provide high quality mathematics/science 
education. Select the response that best describes your overall assessment of the likely effect of this session in each of the 
following areas. 

□ Not applicable (The session did not focus on building capacity for classroom instruction.) 











Mixed or 














Negative 




Neutral 




Positive 


Don’t 








effect 




effect 




effect 


know 


N/A 


1. 


Participants’ ability to identify and understand important 


















ideas of mathematics/science. 


o 


o 


o 


o 


o 


o 


o 


2. 


Participants’ understanding of mathematics/science as a dynamic 


















body of knowledge generated and enriched by investigation. 


o 


o 


o 


o 


o 


o 


o 


3. 


Participants’ understanding of how students learn. 


o 


o 


o 


o 


o 


o 


o 


4. 


Participants’ ability to plan/provide high quality mathematics/ 


















science classroom instruction. 


o 


o 


o 


o 


o 


o 


o 


5. 


Participants’ ability to implement the designated 


















instructional materials. 


o 


o 


o 


o 


o 


o 


o 


6. 


Participants’ self-confidence as mathematics/science instructors. 


o 


o 


o 


o 


o 


o 


o 


7. 


Professional networking among participants with regard to 


















mathematics/science instruction. 


o 


o 


o 


o 


o 


o 


o 



Comments (optional): 
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B. Likely Impact on Participants’ Leadership Capacity 



If the session included any teacher leaders, principals, or others in leadership positions, consider the likely impact of this 
session on their leadership capacity. Select the response that best describes your overall assessment of the likely effect of 
this session in each of the following areas. Please note that even if an element was not addressed explicitly, it might have 
a negative or positive effect on leadership development, depending on whether it was modeled well or poorly. 



□ Not applicable (The session did not include teacher leaders, principals, or others in leadership positions.) 

Mixed or 







Negative 




Neutral 




Positive 


Don’t 








effect 




effect 




effect 


know 


N/A 


1. 


Leaders’ knowledge and understanding of 


















mathematics/science. 


o 


o 


o 


o 


o 


o 


o 


2. 


Leaders’ knowledge and understanding of effective classroom 


















practice. 


o 


o 


o 


o 


o 


o 


o 


3. 


Leaders’ ability to convey to others a vision of effective 


















mathematics/science classrooms. 


o 


o 


o 


o 


o 


o 


o 


4. 


Leaders’ understanding of teachers’ prior knowledge 


















and areas where teachers have difficulty. 


o 


o 


o 


o 


o 


o 


o 


5. 


Leaders’ understanding of adult learners. 


o 


o 


o 


o 


o 


o 


o 


6. 


Leaders’ understanding of the reform process. 


o 


o 


o 


o 


o 


o 


o 


7. 


Leaders’ understanding of important strategies for reform 


















of mathematics/science education. 


o 


o 


o 


o 


o 


o 


o 


8. 


Leaders’ ability to plan/implement exemplary 


















professional development. 


o 


o 


o 


o 


o 


o 


o 


9. 


Leaders’ confidence in serving in leadership roles. 


o 


o 


o 


o 


o 


o 


o 


10. 


Professional networking among leaders with regard to 


















leadership issues. 


o 


o 


o 


o 


o 


o 


o 



Comments (optional): 
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C. Capsule Description of the Quality of the Professional Development Session 

In this final rating of the session, consider all available information about the session, its context and purpose, and your own 
judgment of the relative importance of the ratings you have made. Select the capsule description that best characterizes the 
session you observed. Keep in mind that this rating is not intended to be an average of all the previous ratings, but should 
encapsulate your overall assessment of the quality and likely impact of the session. Please provide a brief rationale for your 
final capsule description of the session in the space provided. 

O Level 1: Ineffective Professional Development 

There is little or no evidence of participant thinking or engagement with important ideas of mathematics/science education. 
Session is unlikely to enhance the capacity of participants to provide high quality mathematics/science education or to be 
effective leaders of mathematics/science education in the district(s). Professional development appears to be either (select 
one below): 

O Passive ‘‘Learning” 

Session is pedantic and uninspiring. Participants are passive recipients of information; material is presented in a way 
that is inaccessible to or inappropriate for many of the participants. 

O Activity for Activity’s Sake 

Participants are involved in hands-on activities or other individual or group work, but it appears to be activity for 
activity’s sake. Session lacks a clear sense of purpose and/or a clear link to the conceptual development of 
participants. 

O Level 2: Elements of Effective Professional Development 

Session contains some elements of effective practice in professional development, but there are substantial problems in the 
design, content, and/or implementation given the purposes of the session. For example, the content is presented in a way 
that would reinforce misconceptions or the pace is clearly too rapid for meaningful participant engagement. Overall, the 
session is quite limited in its likelihood to enhance the capacity of most participants to provide high quality 
mathematics/science education or to be effective leaders of mathematics/science education in the district(s). 

O Level 3: Beginning Stages of Effective Professional Development (Select one below.) 

O Low 3 O Solid 3 O High 3 

Professional development is purposeful and at times effective, but there are some weaknesses in the design, content, or 
implementation of the session. For example, participants’ expertise is not well-utilized; or participants are not given 
sufficient opportunity to reflect on what they are learning. Overall, the session is somewhat limited in its likelihood to 
enhance the capacity of participants to provide high quality mathematics/science education or to be effective leaders of 
mathematics/science education in the district(s). 

O Level 4: Accomplished, Elective Professional Development 

Facilitation is skillful and participants are engaged in purposeful work (e.g., investigations, discussions, presentations, 
reading) designed to deepen their understanding of important mathematics/science concepts; enhance their pedagogical 
skills and knowledge; increase their ability to use the designated instructional materials; or to enhance their leadership 
skills. The facilitator(s) implement the professional development session well and participants’ contributions are valued, 
but adaptation of content or format in response to participants’ needs and interests may be somewhat limited. The session 
is quite likely to enhance the capacity of most participants to provide high quality mathematics/science education or to be 
effective leaders of mathematics/science education in the district(s). 

O Level 5: Exemplary Professional Development 

Facilitation is skillful, and participants are highly engaged in purposeful work (e.g., investigations, discussions, 
presentations, reading) designed to deepen their understanding of important mathematics/science concepts; enhance their 
pedagogical skills and knowledge; increase their ability to use the designated instructional materials; or to enhance their 
leadership skills. The session is artfully implemented, with flexibility and responsiveness to participant needs/interests. 

The session is highly likely to enhance the capacity of participants to provide high quality mathematics/science education 
or to be effective leaders of mathematics/science education in the district(s). 

Please provide your rationale for the capsule rating: 
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